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PREFACE 


The  U.S.  Government  has  invested  heavily  in  sophisticated 
computer  models  to  predict  the  transport  and  dispersion  (T&D) 
and  the  ensuing  human  effects  of  Chemical,  Biological, 
Radiological,  and  Nuclear  (CBRN)  agents  and  materials.  As  new 
modeling  capabilities  are  added,  there  is  a  continuing  need  to 
verify  and  validate  T&D  models  through  rigorous  comparisons  to 
field  trial  data.  Since  2000,  IDA  has  taken  part  in  verification  and 
validation  activities  connected  with  the  Hazard  Prediction  and 
Assessment  Capability  (HPAC)  model  developed  by  the  Defense 
Threat  Reduction  Agency  (DTRA).  In  past  studies  we  have 
compared  the  predictions  of  HPAC  to  several  field  trial  data  sets, 
covering  a  broad  range  of  release  conditions,  terrain,  and  weather 
conditions.  Additional  insight  into  model  performance  and 
capabilities  can  be  gleaned  by  comparing  the  model  predictions 
of  two  or  more  models  to  field  trial  data  and  to  one  another, 
which  is  the  approach  taken  in  the  present  study.  The  second 
model  used  in  this  study  is  the  National  Atmospheric  Release 
Advisory  Center  (NARAC)  model,  developed  by  the  Lawrence 
Livermore  National  Laboratory  (LLNL).  Two  previous 
collaborative  IDA/LLNL  studies  compared  HPAC  and  NARAC. 
The  first  was  an  exhaustive  comparison  of  HPAC  and  NARAC 
predictions  to  the  well-known  Prairie  Grass  field  trial  data.  The 


second  study  compared  HPAC  and  NARAC  predictions  for  17 
hypothetical  releases  (a  model-to-model  comparison).  In  that 
study  there  was  general  good  agreement  between  HPAC  and 
NARAC.  There  were  some  significant  differences  between 
HPAC  and  NARAC  for  elevated  releases,  suggesting  distinct 
modeling  choices  between  the  two  models  for  elevated  releases. 
Prompted  by  these  results  and  by  a  review  of  vertical  transport 
modeling  in  the  NARAC  model  by  Dr.  J.  Weil,  IDA  and  LLNL 
undertook  the  present  study  to  compare  HPAC  and  NARAC  to 
the  Copenhagen  releases  -  a  set  of  ten  elevated  releases 
conducted  in  the  late  seventies.  This  document  presents  the 
details  of  this  comparison. 

The  IDA  participants  were  Dr.  James  F.  Heagy,  Dr.  Steve 
Warner,  and  Dr.  Nathan  Platt;  Dr.  Michael  Dillon  from  LLNL  is 
also  a  co-author.  We  gratefully  acknowledge  useful  discussions 
and  review  from  Dr.  Ian  Sykes  of  L3-Com.  The  IDA  review 
committee  consisted  of  Dr.  Davy  Lo  (SED),  Dr.  Vincent 
(Bram)  Lillard  (OED),  Dr.  Don  Lloyd  (SFRD),  Dr.  Dennis 
DeRiggi  (SED),  and  was  chaired  by  Dr.  George  Koleszar  (SED 
Director).  The  sponsor  for  this  work  was  Richard  N.  Fry 
(DTRA). 
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EXECUTIVE  SUMMARY 


EXECUTIVE  SUMMARY 


A.  BACKROUND 

This  study  is  part  of  a  continuing  effort  to  support  the 
verification  and  validation  activities  associated  with  DTRA’s 
HP  AC  model.  The  specific  area  of  interest  in  this  study  is  that  of 
elevated  releases.  In  a  previous  study  that  compared  HPAC  to 
LLNL’s  NARAC  model  for  a  set  of  hypothetical  releases, 
sizeable  differences  were  observed  between  HPAC  and  NARAC 
for  some  elevated  releases.  In  those  releases  HPAC  significantly 
overpredicted  the  NARAC  ground  dosages;  the  HPAC  plume 
exhibited  a  faster  settling  rate  and  larger  vertical  diffusivity. 
Prompted  by  these  results,  IDA  teamed  with  LLNL  to  undertake 
the  present  study,  which  compares  HPAC  and  NARAC 
predictions  to  a  set  of  ten  elevated  release  experiments  -  The 
Copenhagen  Field  Trials  -  carried  out  in  Copenhagen,  Denmark 
in  the  late  1970s. 

B.  STUDY  FRAMEWORK 

The  design  of  the  study  was  to  run  each  of  the  models  using  as 
much  information  (e.g.,  meteorology,  terrain,  release  parameters) 
as  they  could  ingest,  in  order  to  get  the  best  possible  “scientific” 
predictions.  This  required  a  great  deal  of  coordination  between 
IDA  and  LLNL  during  the  initial  phases  of  this  work.  We 
emphasize  that  this  was  not  an  operational  comparison  between 
the  two  models.  IDA  performed  all  of  the  HPAC  runs,  while 
LLNL  performed  all  of  the  NARAC  runs.  LLNL  provided  IDA 
with  spreadsheets  containing  the  Copenhagen  sampler  data 
(ground  truth  concentration  data),  as  well  as  the  meteorological 
data  for  each  release  (mainly  wind  speeds  and  directions),  which 
they  extracted  from  the  original  Copenhagen  published  literature. 
LLNL  also  sent  all  of  their  predictions  to  IDA  and  IDA 
performed  the  comparison  analyses. 
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C.  OVERVIEW  OF  COPENHAGEN  RELEASES 

The  Copenhagen  releases  comprise  a  set  of  ten  continuous 
release  Sulfur  Hexafluoride  (SF6)  tracer  experiments  conducted 
in  Gladsaxe,  Copenhagen  in  1978  and  1979.  The  tracers  were 
released  from  a  TV  tower  115  m  above  ground  level  (AGL).  All 
releases  were  roughly  1.5  hours  in  duration  with  flow  rates 
ranging  from  2.4  -  4.7  gs"1.  The  downwind  region  of  the  release 
(East  of  the  release)  was  mainly  residential,  with  an  estimated 
surface  roughness  of  0.6  m.  Releases  were  conducted  under 
neutral  boundary  layer  conditions  (3  releases)  and  unstable 
boundary  layer  conditions  (7  releases).  Sampling  arcs  were 
placed  at  rough  distances  of  2  km,  4  km,  and  6  km  from  the 
release  point.  Approximately  20  samplers  per  arc  were 
employed  for  each  release  and  not  all  arcs  were  used  for  all  of 
the  releases.  Average  concentrations  at  each  sampler  were 
measured  over  3  successive  20  minute  sampling  intervals. 

D.  HPAC  TERRAIN/SURFACE  MODES  AND  WEATHER 

INPUTS 


Four  HPAC  terrain  and  surface  type  combinations  were 
investigated,  summarized  in  the  table  below. 

TABLE  1:  HPAC  Terrain  and  Surface  Type  Run  Modes 


Mode 

Terrain 

Surface  Type 

1 

Enabled 

Constant  surface  roughness  =  0.6  m 

2 

Enabled 

Landcover  data 

3 

Flat 

Constant  surface  roughness  =  0.6  m 

4 

Flat 

Landcover  data 

Three  meteorological  input  options  (referred  to  as  modes 
herein)  were  examined  for  the  HPAC  runs  -  a  “Full  Met”  mode 
and  two  “Operational”  excursions. 


The  full  met  mode  employed  all  of  available  meteorological 
inputs  that  could  be  used  by  HP  AC,  namely  wind  speed,  wind 
direction,  and  temperature,  all  at  10-minute  intervals,  as  well  as 
Monin-Obukhov  length1  and  inversion  height.  The  operational 
excursions  employed  only  wind  speed  and  direction  (Excursion 
1)  or  wind  speed,  wind  direction,  and  temperature  (Excursion  2). 
The  full  met  mode  was  used  with  all  four  terrain/surface  modes, 
while  the  operational  excursions  were  used  with  terrain/surface 
Mode  1  only. 

E.  SUMMARY  OF  RESULTS 

Graphical  and  statistical  comparisons  of  HP  AC  and  NARAC 
predictions  to  the  observed  one  hour  average  concentrations  were 
performed.  The  statistical  metrics  employed  were:  Fractional 
Bias  (FB),  Normalized  Absolute  Difference  (NAD),  Normalized 
Mean  Square  Error  (NMSE),  Bounded  Normalized  Mean  Square 
Error  (BNMSE),  and  FACx,  the  fraction  of  prediction  to 
observation  concentration  ratios  within  a  factor  of  x2.  We 
employed  Fac2,  Fac5,  and  FaclO.  All  comparison  metrics  were 
computed  on  a  release-by-release  basis  using  concentrations  that 
were  paired  in  space  and  time  and  for  all  data/prediction  pairs 
within  the  release.  Average  statistics  were  also  computed  by 
averaging  each  of  the  metrics  over  the  releases. 

On  the  whole,  HPAC  performed  best  when  including  terrain 
and  constant  surface  roughness  (Mode  1). 


1  The  height  at  which  buoyant  and  inertial  convective  accelerations  are 
equal. 

2  Excluding  prediction/observation  pairs  having  zero  predictions  or  zero 
observations. 
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The  summary  statements  that  follow  apply  to  HPAC  Mode  1 
with  full  met.  Similar  results  were  found  with  the  other  modes 
and  met  options.  As  measured  by  the  fractional  bias,  HPAC 
underpredicted  all  9  releases  considered  for  analysis  (the  first 
release  was  not  used),  while  NARAC  underpredicted  7  of  the  9 
releases.  HPAC  underpredicted  NARAC  for  all  of  the  releases. 
Table  1  below  shows  comparative  results  for  HPAC  and 
NARAC  for  all  prediction/observations  pairs  used  in  the 
analysis.  HPAC  underpredictions  relative  to  the  data  and  to 
NARAC  are  clear  from  the  table. 

Differences  between  HPAC  and  NARAC  for  all  metrics 
except  Fac2,  Fac5,  and  Fac  10  were  found  to  be  statistically 
significant  at  the  2%  level  (and  at  the  1%  level  using 
multivariate  ANOVA  for  the  combined  metrics  FB,  NAD, 
BNMSE,  Fac2,  Fac5,  and  FaclO). 


TABLE  2:  Selected  Statistical  Comparisons  of  HPAC  and  NARAC 
Predictions  of  the  Elevated  Copenhagen  Releases 


NARAC 

HPAC 

OBSERVATIONS 

Mean  (ng/m3) 

690 

347 

846 

Median  (nq/m3) 

394 

210 

399 

Fac2 

0.419 

0.332 

Fac5 

0.740 

0.732 

FaclO 

0.834 

0.883 

Fraction  underpredicted 

0.570 

0.774 

Fraction  overpredicted 

0.430 

0.226 

Fraction  NARAC  >  HPAC 

0.728 

Fraction  HPAC>  NARAC 

0.272 

Analyses  of  time  sequences  of  HP  AC  concentration  contours 
show  that  the  HP  AC  plume  reaches  the  ground  and  extends  well 
above  the  release  height  in  a  short  time  scale,  suggesting  a  large 
vertical  diffusivity  (perhaps  larger  than  that  of  NARAC).  With 
this  in  mind,  we  speculate  that  the  “missing”  mass  in  the  HP  AC 
simulations  either  remains  lofted  or  reaches  the  ground  before  the 
sampling  arcs  for  these  elevated  releases.  Based  on  this  study, 
we  recommend  that  the  HPAC/SCIPUFF  developer  reexamine 
the  algorithm  and  parameterization  associated  with  the 
representation  of  vertical  diffusivity  for  elevated  releases.  The 
goal  of  such  a  reexamination  would  be  to  simply  identify  and 
trace  the  original  case  for  verification  and  validation  of  these 
elevated  release  features. 

The  major  findings  of  this  study  are  as  follows.  The  main 
finding  is  that  both  HP  AC  and  NARAC  underpredict  the  1-hour 


average  ground  concentrations  of  the  elevated  Copenhagen 
releases,  with  the  HP  AC  underpredictions  being  much  more 
pronounced  than  those  of  NARAC.  With  respect  to  the 
statistics  FB,  NAD,  NMSE,  and  BNMSE,  the  differences 
between  HPAC  and  NARAC  are  statistically  significant  at  the  2 
percent  level.  Multivariate  ANOVA  hypothesis  applied  to  FB, 
NAD,  BNMSE,  Fac2,  Fac5,  and  FaclO  shows  that  HPAC  and 
NARAC  differ  at  the  1%  level  of  significance.  Resampled 
HPAC  and  NARAC  summed  concentration  MOEs  are  also  well 
separated.  The  4  HPAC  terrain  /landcover  modes  and  the  3  met 
options  do  not  “fix”  the  severe  HPAC  underpredictions.  Finally, 
the  HPAC  vertical  slice  concentration  contours  suggest  that  the 
modeled  HPAC  vertical  diffusivity  is  larger  than  what  was 
present  during  the  releases. 
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BRIEFING  REPORT 
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INTRODUCTION 


This  work  was  funded  by  the  Defense  Threat  Reduction 
Agency  (DTRA)  as  part  of  validation  efforts  for  their  Hazard 
Prediction  Assessment  Capability  (HPAC)  model.  IDA  was 
tasked  by  DTRA  to  provide  independent  technical  analyses  of  the 
HPAC  system  in  support  of  the  DTRA  validation  efforts.  The 
work  carried  out  in  this  study  is  the  third  in  a  series  of  works  that 
involve  HPAC  and  the  National  Atmospheric  Release  Advisory 
Center  (NARAC)  model,  developed  by  the  Lawrence  Livermore 
National  Laboratory  (LLNL).  NARAC  currently  serves  as  the 
Department  of  Homeland  Security  (DHS)  hazardous  material 


atmospheric  transport  and  dispersion  model  within  the 
Interagency  Modeling  and  Atmospheric  Assessment  Center 
(IMAAC).  The  purpose  of  the  study  was  to  advance  the  HPAC 
and  NARAC  model  physics  using  field  data  from  the 
Copenhagen  tracer  releases  conducted  in  the  late  seventies.  This 
study  was  not  intended  to  be  an  operational  intercomparison  or 
evaluation  between  the  HPAC  and  NARAC  systems,  as  much  of 
the  information  used  is  typically  not  available  during  an 
operation  response.  The  companion  evaluation  of  the  NARAC 
modeling  system  by  Weil  is  also  available  [Ref.  3], 
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I  DA 


INTRODUCTION 


•  Preface 

-  Institute  for  Defense  Analyses  (IDA)  is  a  non-profit  research  and 
development  center,  serving  the  Office  of  the  Secretary  of  Defense,  Defense 
Agencies,  Unified  Combatant  Commands,  and  the  Joint  Staff. 

-  For  this  task,  we  provide  independent  technical  analyses  to  support  model 
validation  efforts. 

•  Atmospheric  Transport  and  Dispersion  Models  are  Required  by  U.S. 

Government 

-  DOD:  chemical  and  biological  attacks  on  the  battlefield  or  at  military 
facilities 

-  DHS:  terrorist  attacks  on  homeland 

•  Models  Examined  in  this  Study 

-  Hazard  Prediction  and  Assessment  Capability  (HPAC):  DTRA 

-  National  Atmospheric  Release  Advisory  Center  (NARAC)  Model :  DHS/LLNL- 
DOE 
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BRIEF  MODEL  DESCRIPTIONS 


The  HP  AC  and  NARAC  methodologies  are  summarized  in 
the  slide.  The  modeling  approaches  for  the  two  models  are  quite 
distinct  -  HPAC  uses  a  puff  formulation,  while  NARAC  uses  a 
particle  propagation  formulation.  Additionally,  meteorological 
data  are  assimilated  by  unique  sub-models  for  the  2  models. 


Version  numbers  of  the  models  and  sub-models  used  in  this 
comparison  are  also  listed.  Both  models  have  been  used 
extensively  in  their  respective  arenas.  Additional  descriptions  of 
these  models  can  be  found  in  IDA  papers  P-3554,  P-3555, 
references  [1]  and  [2],  respectively  on  the  next  slide. 
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BRIEF  MODEL  DESCRIPTIONS 


•  HPAC  -  Version  4.04,  Service  Pack  3  (4.04.012) 

-  Second-Order  Closure  Integrated  Puff  Model  (SCI PUFF) 

•  Lagrangian  model  that  uses  Gaussian  puff  method;  turbulent  dispersion  parameterization  is 
based  on  second-order  closure  theory,  allowing  for  estimation  of  concentration  mean  and 
variance 

•  Version  2.2 

-  Stationary  Wind  Fit  and  Turbulence  (SWIFT) 

•  Mass-consistent  wind  field  model 

-  Applied  to  military  planning  and  various  national  defense  problems 

•  NARAC 

-  Lagrangian  Operational  Dispersion  Integrator  (LODI) 

•  Lagrangian  Monte  Carlo  model:  Solves  3D  advection-diffusion  equation  by  integrating  a 
stochastic  differential  equation 

•  Version  1.0  2003  Dec23  (Random  number  seed  79836158.0000000) 

-  Atmospheric  Data  and  Parameterization  Tool  (ADAPT) 

•  Data  assimilation  tool  that  constructs  meteorological  fields  using  a  variety  of  interpolation 
techniques  and  atmospheric  parameterizations 

•  Version  DEV2.12 

-  Serves  as  the  primary  atmospheric  transport  and  dispersion  modeling  system  for  the 
Department  of  Homeland  Security  (DHS)  Interagency  Modeling  &  Atmospheric 
Assessment  Center  (IMAAC)  and  Department  of  Energy  (DOE)  Atmospheric  Release 
Advisory  Center  (ARAC) 


7 


PREVIOUS  HPAC-NARAC  COMPARISONS 


Two  previous  studies  that  compare  HP  AC  and  NARAC  are 
listed  in  the  slide.  The  Prairie  Grass  field  trials  were  examined 
in  an  extensive  comparison  in  Warner  S.,  Platt,  N.,  Heagy,  J.  F., 
Bradley,  S.,  Bieberbach,  G.,  Sugiyama,  G.,  Nasstrom,  J.  S., 
Foster,  K.  T,  and  Larson,  D.,  2001:  User-Oriented  Measures  of 
Effectiveness  for  the  Evaluation  of  Transport  and  Dispersion 
Models,  IDA  Paper  P-3554,  797  pp,  January  2001  (reference  [1]). 
Both  HP  AC  and  NARAC  compared  favorably  to  these  short- 
range  field  trials,  performing  best  for  unstable  releases.  NARAC 
was  found  to  be  somewhat  more  conservative  than  HP  AC  for  low 
threshold  dose  comparisons  (NARAC  displayed  fewer  false 
negatives  and  more  false  positives). 


HP  AC  and  NARAC  were  also  compared  in  a  model-to- 
model  study:  Warner,  S.,  Heagy,  J.  F.,  Platt,  N.,  Larson,  D., 
Sugiyama,  G.,  Nasstrom,  J.  S.,  Foster,  K.  T.,  Bradley,  S.,  and 
Bieberbach,  G.,  2001:  Evaluation  of  Transport  and  Dispersion 
Models:  A  Controlled  Comparison  of  Hazard  Prediction  and 
Assessment  Capability  (HP AC)  and  National  Atmospheric 
Release  Advisory  Center  (NARAC)  Predictions,  IDA  Paper  P- 
3555,  251  pp,  May  2001  (reference  [2]).  Seventeen  releases 
were  analyzed  with  good  overall  agreement  found  between 
HP  AC  and  NARAC.  Slight  model  differences  were  observed  as 
functions  of  particle  size  and  atmospheric  stability.  Some 
significant  differences  (shown  in  the  next  two  slides)  were 
observed  as  a  function  of  release  height. 
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I  DA  PREVIOUS  HP  AC  -  NARAC  COMPARISONS 


•  Prairie  Grass  Field  Experiment 

-  Similar  model  performance  -  both  models  performed  best  for  releases  that  occurred 
during  unstable  conditions 

-  NARAC  somewhat  more  ’’conservative”  (overpredictive)  than  HPAC  with  respect  to 
predicting  hazard  regions  (i.e.,  samplers  at  which  a  low  threshold  is  exceeded) 

•  Fewer  false  negatives 

•  Higher  false  positives 

-  [1]  WarnerS.,  Platt,  N.,  Heagy,  J.  F.,  Bradley,  S.,  Bieberbach,  G.,  Sugiyama,  G.,  Nasstrom, 
J.  S.,  Foster,  K.  T,  and  Larson,  D.,  2001:  User-Oriented  Measures  of  Effectiveness  for  the 
Evaluation  of  Transport  and  Dispersion  Models,  IDA  Paper 

P-3554,  797  pp,  January  2001. 

•  “Model-to-Model”  Comparisons  for  17  Simple  Releases 

-  Overall,  very  close  agreement,  but  only  after  carefully  ensuring  the  comparability  of  input 
parameters  and  model  settings 

-  Differences  detected  as  a  function  of 

•  particle  size  (slight  differences) 

•  atmospheric  stability  (slight  differences) 

•  release  height  (some  significant  differences) 

-  [2]  Warner,  S.,  Heagy,  J.  F.,  Platt,  N.,  Larson,  D.,  Sugiyama,  G.,  Nasstrom,  J.  S., Foster,  K. 
T.,  Bradley,  S.,  and  Bieberbach,  G.,  2001:  Evaluation  of  Transport  and  Dispersion  Models: 
A  Controlled  Comparison  of  Hazard  Prediction  and  Assessment  Capability  (HPAC)  and 
National  Atmospheric  Release  Advisory  Center  (NARAC)  Predictions,  IDA  Paper  P-3555, 
251  pp,  May  2001. 
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PREVIOUS  MODEL-TO-MODEL  STUDY-EXAMPLE  DIFFERENCES  IN  SURFACE 
DOSAGE  FOR  HPAC  &  NARAC  PREDICTIONS  OF  ELEVATED  RELEASES 


The  slide  shows  surface  dosage  contours  for  HPAC  (Blue  — ) 
and  NARAC  (Red  -)  for  two  elevated  releases  (MvM  3  and  MvM 
15)  considered  in  the  model-to-model  study  [2],  MvM  3  was  a 
gas  release  (SF6)  carried  out  under  stable  conditions  with  a 
boundary  layer  height  of  100  m  and  release  height  of  80  m,  while 


MvM  15  was  a  particle  release  carried  out  under  neutral 
conditions  with  a  boundary  layer  height  of  500  m  and  release 
height  of  750  m.  In  each  case  the  HPAC  surface  dosage 
contours  cover  much  larger  areas  than  corresponding  NARAC 
dosage  contours  (NARAC  predicts  lower  surface  concentrations 
than  HPAC  for  these  releases). 
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PREVIOUS  MODEL-TO-MODEL  STUDY 

Example  Differences  in  Surface  Dosage  for 
HPAC  &  NARAC  Predictions  of  Elevated  Releases 


HPAC  (Blue  -)  and  NARAC  (Red  -)  surface  dosage  contours  for  two 
elevated  release  scenarios:  MvM  3  at  30  and  60  Minutes  and  MvM  15  at 
120  and  180  minutes.  Each  release  shows  significant  NARAC 
underpredictions  with  respect  to  HPAC  (from  reference  [2]). 


MvM  3 

1  kg  instantaneous  “gas”  release 
Stable  boundary  layer 
Release  height  =  80  m 
Boundary  layer  height  =100  m 


MvM03:  Dosage  (kg  s/m3),  Height  =  10  m,  Time  =  30  min.  MvM03:  Dosage  (kg  s/m3),  Height  =  10  m,  Time  =  60  min. 


MvM  15 

1  kg  instantaneous  “particle”  release 
Neutral  boundary  layer 
Release  height  =  750  m 
Boundary  layer  height  =  500  m 


MvM15:  Dosage  (kg  s/m3),  Height  =  10  m,  Time  =  120  min. 


MvM15:  Dosage  (kg  s/m3),  Height  =  10  m,  Time  =  180  min. 
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PREVIOUS  MODEL-TO-MODEL  STUDY  NARAC  AND  HPAC  INSTANTANEOUS 
CONCENTRATIONS:  VERTICAL  SLICE  FOR  ELEVATED  RELEASE  MVM  3 

The  figure  in  the  slide  shows  a  comparison  of  HPAC  and  and  enhanced  vertical  dispersion  with  respect  to  the  NARAC 
NARAC  concentration  contours  over  time  for  the  elevated  contours,  indicating  distinct  modeling  choices  for  the  two 
release  MvM  3  [2],  The  HPAC  contours  show  enhanced  settling  models. 
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PREVIOUS  MODEL-TO-MODEL  STUDY 

NARAC  and  HPAC  Instantaneous  Concentrations: 
Vertical  Slice  for  Elevated  Release  MvM  3 


I  DA 


NARAC  and  HPAC  10  9  kg/m3  (1000  ng/m3)  instantaneous  concentration 
contours  at  t-  10,  20,  30,  40,  50,  and  60  minutes.  The  material  is  moving 
from  right  to  left  The  HPAC  contours  show  much  greater  vertical  diffusion 
and  descend  at  a  greater  rate  (from  reference  [2]). 


MvM03:  Concentration  (kg/m3), Vertical  Slice  at  y  =  0  km 


Downwind  (km) 
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RECENT  ANALYSES  SUGGEST  NEED  FOR  ELEVATED  RELEASE  EXAMINATIONS 


Recent  studies  with  HP  AC  and  NARAC  suggest  there  is  a 
need  for  enhanced  modeling  capabilities  for  elevated  releases  for 
both  models.  Comparisons  of  HP  AC  to  the  Over-Land  Wind 
Dispersion  (OLAD)  field  experiment  data  showed  under¬ 
predictions  for  some  of  the  elevated  releases.  A  recent  review  of 
NARAC  (Weil,  J.  C.,  Review  of  Approaches  and  Data  Sets  for 
Evaluation  of  the  NARAC  Modeling  System,  Cooperative 
Institute  in  Environmental  Sciences  Report  no.  UCRL-TR- 
2011989,  September  2003),  identified  the  need  for  more 
complete  testing  of  NARAC  vertical  dispersion  formulations. 
This  report  specifically  recommends  comparisons  of  NARAC  to 
the  Copenhagen  Tracer  Experiments.  The  recommended 


evaluation  of  the  NARAC  modeling  system  with  the 
Copenhagen  field  data  was  performed.  In  summary,  the 
NARAC  predictions  were  in  good  overall  agreement  (60  percent 
of  the  NARAC  predictions  were  within  a  factor  of  2  of  the 
observations  after  accounting  for  the  meteorological 
uncertainty)  See  [3]  for  more  details  including  a  discussion  of 
the  NARAC  vertical  dispersion  parameterization.  We  note  that 
the  Copenhagen  releases  were  conducted  under  unstable  or 
neutral  conditions  with  release  heights  well  below  the  boundary 
layer  height.  Therefore,  the  present  study  is  unlikely  to  explain 
the  previously  noted  model-to-model  differences.  However,  we 
expect  the  results  of  this  study  will  further  characterize  this 
issue. 
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RECENT  ANALYSES  SUGGEST  NEED  FOR 
ELEVATED  RELEASE  EXAMINATIONS 


•  Over-Land  Along-Wind  Dispersion  (OLAD)  Field  Experiment 

-  HPAC  under-predictions  of  some  elevated  releases 

•  Recent  Review  for  NARAC  Evaluation  Approaches 

-  [3]  Weil,  J.  C.,  Review  of  Approaches  and  Data  Sets  for  Evaluation  of  the  NARAC 
Modeling  System,  Cooperative  Institute  for  Research  in  Environmental  Sciences 
Report  no.  UCRL-TR-201989,  September  2003. 

-  Identifies  several  attributes  of  field  experiments  that  would  support  future  NARAC 
evaluation 

-  Calls  for  more  complete  testing  of  vertical  dispersion  formulations 

-  Specifically  recommends  Copenhagen  Tracer  Experiments  of  1978/1979 

•  Weil,  J.  C.,  Evaluation  of  the  NARAC  Modeling  System:  Final  Report  to  the 
Lawrence  Livermore  National  Laboratory,  Cooperative  Institute  for  Research  in 
Environmental  Sciences  Report  no.  UCRL-TR-217329,  September  2004. 

-  Evaluated  NARAC  with  both  Copenhagen  field  data  and  a  Large  Eddy  Simulation 
particle  model 

-  Demonstrated  overall  good  agreement  between  NARAC  predictions  and  Copenhagen 
field  data  (60%  of  the  NARAC  predictions  were  within  a  factor  of  2  of  the  observations 
after  accounting  for  wind  direction  uncertainty) 

-  Evaluated  NARAC  vertical  dispersion  parameterization:  overall  parameterization  was 
reasonable,  although  for  elevated  releases  under  unstable  atmospheric  conditions 
NARAC  modestly  underpredicts  surface  concentrations  near  the  source 
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OVERVIEW  OF  COPENHAGEN  RELEASES 


The  releases  considered  in  this  study  were  conducted  in  the 
Gladsaxe  region  of  Copenhagen  and  cover  a  period  from  14 
September  1978  though  19  July  1979  [refs.  4,  5,  6],  Ten  separate 
continuous  releases  of  the  tracer  gas  Sulfur  Hexafluoride  (SF6) 
were  carried  out.  Release  durations  were  all  approximately  1.5 
hours  and  SF6  flow  emission  rates  ranged  from  2.4  gs'1  to 
4.7  gs'1.  All  sources  were  situated  on  a  TV  tower  115  meters 
above  ground  level  (AGL).  The  downwind  region  of  the  release 
site  (mainly  East  of  the  releases)  was  residential,  with  an 
estimated  surface  roughness  of  0.6  m  [4],  Three  releases  were 
conducted  under  neutral  boundary  layer  conditions,  while  7  were 
conducted  under  unstable  boundary  layer  conditions.  Weather 


measurements  were  taken  at  several  heights  along  the  TV  tower; 
measurements  included  temperature,  wind  speed,  and  wind 
direction  at  10-minute  intervals.  Hourly  averaged  vertical  and 
cross-wind  wind  speed  fluctuations  were  also  measured  near  the 
point  of  the  releases.  Three  “arcs”  of  SF6  samplers  were 
employed  in  the  experiments  at  distances  of  roughly  2  km,  4 
km,  and  6  km  from  the  release  location.  Each  sampling  unit 
contained  3  plastic  bags  that  were  inflated  sequentially  in  20- 
minute  sampling  periods,  for  a  net  sampling  period  of  one  hour 
per  sampler.  Details  of  the  sampling  methodology  can  be  found 
in  [4]  and  [5].  Approximately  20  samplers  per  arc  were 
employed  in  the  releases  and  not  all  arcs  were  used  for  all  of  the 
releases. 
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I  DA  OVERVIEW  OF  COPENHAGEN  RELEASES 

The  Copenhagen  releases  comprised  10  continuous  release  SF6  tracer 
experiments  conducted  in  Gladsaxe  Copenhagen,  spanning  14  Sep.  1978 
to  19  July  1979.  Release  durations  were  roughly  1.5  hour,  with  flow 
emission  rates  ranging  from  2.4  -  4.7  gs1. 

All  tracers  were  released  from  a  TV  tower  at  115  m  AGL.  The  downwind 
region  of  the  release  (East  of  the  release)  was  mainly  residential,  with  an 
estimated  surface  roughness  of  0.6  m. 

Releases  were  conducted  under  neutral  boundary  layer  conditions  (3 
releases)  and  unstable  boundary  layer  conditions  (7  releases). 

Temperature,  wind  speed  and  direction  measurements  were  taken  at 
various  heights  along  the  tower  at  10  minute  intervals.  Hourly  averaged 
vertical  and  lateral  wind  fluctuations  were  also  measured  near  the 
release  point. 

Sampling  arcs  were  placed  at  rough  distances  of  2  km,  4  km,  and  6  km 
from  the  release  point.  Approximately  20  samplers  per  arc  were 
employed  for  each  release.  Not  all  arcs  were  used  for  all  of  the  releases. 

Average  concentrations  at  each  sampler  were  measured  over  3 
successive  20  minute  sampling  intervals. 
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COPENHAGEN  REFERENCES 


The  three  main  Copenhagen  release  references  are  listed  on  the  slide. 
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COPENHAGEN  REFERENCES 


[4]  Gryning,  Sven-Erik,  Elevated  Source  SF6-Tracer  Dispersion 
Experiments  in  the  Copenhagen  Area ,  Ph.D.  Thesis, 
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Tracer  Experiments  and  Model  Calculations,  Journal  of 
Climate  and  Applied  Meteorology  23,  651-660,  April  1984. 

[6]  Gryning,  Sven-Erik,  Lyck,  E.,  The  Copenhagen  Tracer 
Experiments:  Reporting  of  Measurements,  Rise  National 
Laboratory,  Riso-R-1054,  August  2002. 
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RELEASE  SPECIFICS 


The  release  location  is  shown  in  this  4  km  x  4  km  overhead 
photograph,  which  also  shows  the  residential  area  east 
(downwind)  of  the  release  point.  Since  the  image  date  is  2005, 
details  of  the  downwind  region  are  not  expected  to  match  those 


present  in  1978  and  1979.  Quantitative  details  of  the  10 
releases  are  shown  in  the  table.  Local  Copenhagen  time  is  two 
hours  later  than  UTC,  so  with  the  exception  of  the  first  (9/14/78) 
release,  the  releases  were  carried  out  in  early  to  mid  afternoon. 
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RELEASE  SPECIFICS 


I  DA 

Release  location 

UTM  :  (342580  m  Easting,  6179610  m  Northing), 
Zone  33  =  (55.7366°  N,  12.4927°  E) 

Release  height:  115  m  AGL 

Residential  area:  surface  roughness  =  0.6  m  [4] 
(Note:  a  surface  roughness  of  0.5  m  is  associated 
with  mixed  forest,  towns  and  cities) 

Note:  2005  overhead  photo 
Dimensions:  4  km  x  4  km 


Number 

Date 

Release 

start 

(UTC) 

Release 

stop 

(UTC) 

Release 

duration 

(hr:min) 

Sampling 

start 

(UTC) 

Sampling 
stop  (UTC) 

Sampling 

duration 

(hr:min) 

Tracer 

release 

rate 

(g/s) 

1 

9/14/78 

14:00 

15:25 

1:25 

14:23 

15:23 

1:00 

4.70 

2 

9/20/78 

11:45 

13:17 

1:32 

12:17 

13:17 

1:00 

3.20 

3 

9/26/78 

10:21 

11:40 

1:19 

10:40 

11:40 

1:00 

3.20 

4 

10/19/78 

10:47 

12:18 

1:31 

11:13 

12:13 

1:00 

3.20 

5 

11/3/78 

12:02 

13:21 

1:19 

12:20 

13:20 

1:00 

2.30 

6 

11/9/78 

11:46 

13:27 

1:41 

12:26 

13:26 

1:00 

3.20 

7 

4/30/79 

11:41 

13:02 

1:21 

12:02 

12:42 

0:40 

3.10 

8 

6/27/79 

11:25 

12:45 

1:20 

11:45 

12:45 

1:00 

2.40 

9 

7/6/79 

11:30 

12:55 

1:25 

11:50 

12:50 

1:00 

3.00 

10 

7/19/79 

10:50 

12:20 

1:30 

11:15 

12:18 

1:03 

3.30 
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SAMPLER  LAYOUT  -  ALL  SAMPLERS 


The  figure  shows  the  Copenhagen  sampler  locations  with 
Easting  (x)  and  Northing  (y)  coordinates  relative  to  the  release 
point  (0,  0).  The  three  sampler  arcs  are  roughly  2  km,  4  km,  and 
6  km  from  the  release  location.  Approximately  20  samplers  per 


arc  were  used  for  each  release  and  not  all  arcs  were  active  for  all 
of  the  releases.  The  UTM  coordinates  of  the  samplers  are  given 
in  a  backup  slide.  Activated  samplers  for  each  release  are  also 
pictured  in  a  backup  slide. 
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SAMPLER  LAYOUT  -  ALL  SAMPLERS 


All  Sampler  Locations  Relative  to  Release 
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METEOROLOGICAL  OVERVIEW  OF  RELEASES 


The  table  at  the  top  of  the  slide,  adapted  from  Table  1  in  [5], 
shows  the  overall  meteorological  conditions  for  the  10  releases. 
The  Monin-Obukhov  length  (L),  inversion  height  (zi),  friction 
velocity  (not  listed  in  the  table),  and  Pasquill  stability  class  for 
the  releases  are  derived  parameters.  Details  for  their  origin  are 
described  in  [5].  The  derived  Pasquill  stability  class  appears  to 
be  inconsistent  with  the  zi/L  ratio  for  some  of  the  releases.  For 
example,  release  10,  with  a  zi/L  ratio  of -5.5,  has  an  assigned 
stability  class  of  D  (neutral),  even  though  release  6,  with  a  zi/L 


ratio  of -1.4,  is  declared  class  C  (slightly  unstable).  The  box  at 
the  bottom  of  the  slide  summarizes  the  meteorological 
measurements  and  their  altitudes  above  ground  level  (AGL) 
taken  along  the  TV  tower.  Wind  speed,  wind  direction,  and 
temperature  data  were  taken  at  10-minute  intervals  at  the 
indicated  altitudes.  Vertical  and  cross-wind  wind  speed 
fluctuation  measurements  were  one  hour  averages.  All 
meteorological  data  for  the  HP  AC  and  NARAC  runs  were 
extracted  from  data  tables  in  reference  [6], 
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I  DA  METEOROLOGICAL  OVERVIEW  OF  RELEASES 


Mean  Wind 
Speed  (m/s) 

Hourly  Averaged 
Wind 

Fluctuations 

Number 

Date 

115m 

10  m 

ctv  (m/s) 

ctw  (m/s) 

L:  Monin- 

Obukhov 

Length  (m) 

z,:  Inversion 
Height  (m) 

zJL 

Pasquill  Stability  Class 

i 

9/14/78 

8.9 

- 

1.14 

0.68 

- 

- 

- 

D:  Neutral 

2 

9/20/78 

3.4 

2.1 

0.98 

0.83 

-46 

1980 

-43.0 

C:  Slightly  Unstable 

3 

9/26/78 

10.6 

4.9 

1.39 

1.07 

-384 

1920 

-5.0 

C:  Slightly  Unstable 

4 

10/19/78 

5.0 

2.4 

0.85 

0.68 

-108 

1120 

-10.4 

C:  Slightly  Unstable 

5 

11/3/78 

4.6 

2.5 

0.47 

0.47 

-173 

390 

-2.3 

C:  Slightly  Unstable 

6 

11/9/78 

6.7 

3.1 

0.77 

0.71 

-577 

820 

-1.4 

C:  Slightly  Unstable 

7 

4/30/79 

13.2 

7.2 

2.26 

1.33 

-569 

1300 

-2.3 

D:  Neutral 

8 

6/27/79 

7.6 

4.1 

1.61 

0.87 

-136 

1850 

-13.6 

B-C:  Moderately  Unstable 

9 

7/6/79 

9.4 

4.2 

1.35 

0.72 

-72 

810 

-11.3 

B-C:  Moderately  Unstable 

10 

7/19/79 

10.5 

5.1 

1.71 

0.98 

-382 

2090 

-5.5 

D:  Neutral 

Table  adapted  from  reference  [5];  Friction  velocity  for  each  release  also  estimated 
Release  height  for  all  releases:  115  m  AG L 

Meteorological  measurements  taken  along  TV  tower 

•  Temperature 

-  Heights:  2m,  40m,  80m,  120m,  160m,  200m  AGL 

-  10  minute  intervals 

•  Wind  speed 

-  Heights:  10m,  60m,  120m,  200m  AGL 

-  10  minute  intervals 

•  Wind  direction 

Heights:  10m,  120m,  200m  AGL 

-  10  minute  intervals 

•  Crosswind  (sv)  and  vertical  (sw)  wind  fluctuations 

-  Height:  115m  AGL  (release  height) 

-  One  hour  averages 
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NARAC  MODEL  RUN  PROTOCOLS 


The  NARAC  runs  for  this  study  were  performed  by  Lawrence 
Livermore  National  Laboratory  (LLNL)  researchers.  All  releases 
were  modeled,  except  for  release  1  (9/14/78),  which  was  omitted 
because  of  a  lack  of  consistent  input  meteorology  data  relative  to 
the  other  releases  (complicating  a  best-physics  analysis).  One 
hour  average  SF6  concentrations  were  computed  over  the 
collection  times  given  in  the  table  in  slide  11;  concentrations 
were  computed  at  the  sampler  locations  and  on  an  approximate 
20  km  x  20  km  horizontal  grid  with  100  m  grid  resolution.  The 
10-minute  wind  speed  and  direction  data  were  averaged  over  one 
hour  (to  be  consistent  with  the  sigma  v  and  sigma  w  averaging 


times)  and  these  one  hour  averages  were  used  for  the  NARAC 
input  winds.  Other  NARAC  meteorological  inputs  were: 
friction  velocity,  Monin-Obukhov  length,  surface  roughness 
(0.6  m),  inversion  height,  and  lateral  wind  fluctuations  (av). 
Parameters  that  were  not  used  included  vertical  wind 
fluctuations  (ctw),  Pasquill  stability  class  (this  information  is 
duplicative  after  specifying  the  Monin-Obukhov  length),  and 
temperature.  For  simplicity,  terrain  was  not  used  as  an  input  for 
the  NARAC  calculations  reported  here  (there  was  negligible 
difference  between  runs  with  and  without  terrain).  The 
averaging  time  for  the  NARAC  calculations  was  set  to  one  hour. 
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D A  NARAC  MODEL  RUN  PROTOCOLS 


•  Model  runs  performed  by  LLNL 

-  All  releases  modeled  except  for  09/14/78  release 

-  1  hour  average  SF6  concentrations  computed  at  sampler  locations  and  on  ~ 
20  km  x  20  km  horizontal  grid  with  100  m  grid  resolution 

•  Meteorological  inputs 

-  Wind  speed  and  direction  (1  hour  averages) 

-  Friction  velocity 

-  Monin-Obukhov  (MO)  length 

-  Surface  roughness  (0.6  m) 

-  Inversion  height 

-  Sigma  v  (lateral  wind  fluctuations) 

•  Not  used: 

-  Sigma  w  (vertical  wind  fluctuations) 

-  Pasquill  stability  class  (this  information  is  duplicative  after  specifying  the 
Monin-Obukhov  length) 

-  Temperature 

•  Terrain  not  used 

•  One  hour  averaging  times 
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HPAC  MODEL  RUN  PROTOCOLS 


The  HPAC  runs  for  this  study  were  performed  by  IDA 
research  staff.  All  releases  were  modeled  except  for  release  1 
(9/14/78).  This  release  could  not  be  modeled  because  of  an 
invalid  weather  profile  (.prf  file)  [error  #  99  in  the  .log  file  from 
the  SCIPUFF  Weather  Input  Model  (SWIM)].  We  speculate  that 
the  cause  of  this  error  is  the  absence  of  any  co-located  wind  speed 
and  wind  direction  data  along  the  tower;  this  did  not  occur  for 
any  of  the  other  releases.  Twenty-minute  average  SF6 
concentrations  as  well  as  one-hour  concentrations  were  computed 
at  the  sampler  locations  and  on  the  NARAC  grid  discussed  in  the 
previous  slide.  HPAC  runs  employed  the  raw  10-minute  wind 
speed,  wind  direction,  and  temperature  data.  Other  HPAC 


meteorological  inputs  were:  Monin-Obukhov  length,  surface 
roughness  (0.6  m),  inversion  height,  and  Pasquill  stability  class. 
Parameters  that  were  not  used  included  friction  velocity  (HPAC 
does  not  accept  this  as  input),  vertical  wind  fluctuations  (aw), 
and  lateral  wind  fluctuations  (av).  While  HPAC  does  accept  aw 
and  av  as  inputs,  it  does  so  only  for  preparation  of  hazard  areas, 
not  concentration  outputs.  Terrain  was  used  as  an  input  for  the 
HPAC  calculations,  however  terrain  was  toggled  off  for  some 
excursion  runs.  The  averaging  time  for  all  HPAC  calculations 
was  set  to  20  minutes;  this  matches  the  collection  time  (3 
sampler  bags  per  sampler,  20-minutes  each)  [4,  5]. 
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I  DA  HP  AC  MODEL  RUN  PROTOCOLS 


•  Model  runs  performed  by  IDA 

-  All  releases  modeled  except  for  09/14/78  release 

*  SCIPUFF  Weather  Input  Model  (SWIM)  reports  invalid  weather  profile  (.prf  file);  error  # 
99  in  .log  file 

•  Error  possibly  due  to  absence  of  any  co-located  wind  speed  and  wind  direction  data 

-  20  minute  average  concentrations  and  1  hour  average  concentrations  computed 

•  Meteorological  inputs 

-  Wind  speed  and  direction  (10  minute  updates) 

-  Temperature  (10  minute  updates) 

-  MO  length 

-  Pasquill  stability  class  (overridden  by  MO  length,  when  MO  is  also  used) 

-  Surface  roughness 

-  Inversion  height 

•  Not  used : 

-  Friction  velocity  (HPAC  cannot  accept) 

-  Sigma  v  and  Sigma  w  (fluctuation  inputs  used  only  for  creation  of  HPAC 
hazard  areas) 

•  Terrain  used  (excursion  runs  performed  without  terrain) 

•  20  minute  averaging  times  (matches  collection  times) 
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FOUR  HP  AC  TERRAIN/  SURFACE  MODES  EXAMINED 


Four  HP  AC  terrain  and  surface  mode  combinations  were 
investigated.  Modes  1  and  2  employed  terrain,  while  modes  3 
and  4  did  not.  Modes  1  and  3  used  a  constant  surface  roughness 
of  0.6  m,  while  modes  2  and  4  used  the  HP  AC  internal  landcover 
database.  One  finding  uncovered  in  these  investigations  was  that 
when  HP  AC  sampler  coordinates  for  Mode  3  (no  terrain,  surface 
roughness  =  0.6  m)  were  entered  in  UTM  coordinates,  the  output 
concentrations  (in  the  .smp  file)  were  zero  at  all  sampler  locations 
and  all  times.  The  other  three  modes  did  not  display  this 
behavior.  Converting  the  Mode  3  sampler  coordinates  to  latitude 


and  longitude  alleviated  the  zero  output  problem,  although 
without  having  access  to  the  HPAC  source  code,  it  is  not 
possible  to  know  the  exact  representation  of  the  internal  UTM 
to  lat/lon  conversion  that  HPAC  employs.  Sampler  conversions 
were  carried  out  with  GEOTRANS  2.2.6,  a  National  Geospatial 
Intelligence  Agency  (NGA)  conversion  utility  available  at 
http://earth-info.nima.mil/GandG/geotrans.  It  would  be 
useful  to  allow  UTM  coordinates  to  be  used  for  all  sampler  files, 
irrespective  of  terrain  or  landcover  usage,  and  we  suggest  that 
the  HPAC  developer  consider  adding  this  functionality.2 


2  According  to  the  HPAC  developer  this  functionality  exists,  however  there 
is  no  mention  of  it  in  the  HPAC/SCIPUFF  documentation. 
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FOUR  HP  AC  TERRAIN/  SURFACE 
MODES  EXAMINED 


I  DA 


•  Mode  1:  Terrain  on,  Surface  roughness  =  0.6  m 

•  Mode  2:  Terrain  on,  Landcover  enabled 

•  Mode  3:  Terrain  off,  Surface  roughness  =  0.6  m 

•  Mode  4:  Terrain  off,  Landcover  enabled 

•  HPAC  output  sampler  specifications 

-  UTM  sampler  coordinates  used  for  both  terrain  “on”  cases  and  terrain  “off”, 
landcover  “enabled”  case 

-  UTM  coordinates  do  not  work  with  terrain  “off”,  surface  roughness  =  0.6  m 
case;  concentration  outputs  in  .smp  file  are  all  zeros 

•  Lat/lon  coordinates  required  to  get  nonzero  output 

•  Lat/lon  conversions  made  with  GEOTRANS  2.2.6  (NGA  utility,  available  at 
http://earth-info.nima.mil/GandG/geotrans/) 

•  Unknown  whether  internal  HPAC  conversions  match  those  of  GEOTRANS  2.2.6 

•  Recommend  HPAC  add  UTM  functionality  with  terrain  off 

-  Have  recently  learned  that  this  capability  does  exist  in  HPAC,  although  there  is  no  mention  of  it  in 
the  HPAC/SCPIFF  technical  documentation 
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THREE  HPAC  METEOROLOGICAL  INPUT  OPTIONS  EXAMINED 


Three  meteorological  input  options  were  examined  for  the 
HPAC  runs  -  a  “Full  Met”  mode  and  two  “Operational” 
excursions.  The  full  met  mode  employed  all  of  available 
meteorological  inputs  that  could  be  used  by  HPAC,  namely  wind 
speed,  wind  direction,  and  temperature,  all  at  10-minute  intervals, 
as  well  as  Monin-Obukhov  length,  and  inversion  height.  Pasquill 
stability  information  was  also  entered  into  the  HPAC  profiles 


(.prf  files),  however,  according  to  the  SCIPUFF  Technical 
Documentation  [7],  this  is  ignored  when  the  Monin-Obukhov 
length  is  also  entered.  The  operational  excursions  employed 
only  wind  speed  and  direction  (Excursion  1)  or  wind  speed, 
wind  direction,  and  temperature  (Excursion  2).  The  full  met 
mode  was  used  with  all  four  terrain/surface  modes,  while  the 
operational  excursions  were  used  with  terrain/surface  Mode  1 
only. 
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THREE  HP  AC  METEOROLOGICAL  INPUT 

OPTIONS  EXAMINED 


•  “Full”  meteorology 

-  Wind  speed  and  direction  at  10  minute  intervals 

-  Temperature  at  10  minute  intervals 

-  Inversion  height 

-  Monin-Obukhov  length 

-  PGT  stability  category  (overridden  by  MO  length) 

-  Employed  with  all  4  HP  AC  terrain/surface  modes 

•  Excursion  1:  “Operational”  meteorology 

-  Wind  speed  and  direction  at  10  minute  intervals 

-  Employed  with  terrain  on,  SR=0.6  m  (Mode  1)  only 

•  Excursion  2:  “Operational  with  temperature”  meteorology 

-  Wind  speed  and  direction  at  10  minute  intervals 

-  Temperature  at  10  minute  intervals 

-  Employed  with  terrain  on,  SR=0.6  m  (Mode  1)  only 
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STATISTICAL  COMPARISON  METRICS  EMPLOYED 


The  slide  shows  five  comparison  metrics  employed  in  our 
analyses,  where  CP  stands  for  predicted  concentration  and  C0 
stands  for  observed  concentration.  These  metrics  are:  Fractional 
Bias  (FB),  Normalized  Absolute  Difference  (NAD),  Normalized 
Mean  Square  Error  (NMSE),  Bounded  Normalized  Mean  Square 
Error  (BNMSE),  and  FACx,  the  fraction  of  prediction  to 
observation  concentration  ratios  within  a  factor  of  x,  excluding 


prediction/observation  pairs  with  zero  predictions  or  zero 
observations.  We  employ  Fac2,  Fac5,  and  FaclO  in  this  study. 
In  this  study  all  comparison  metrics  are  computed  on  a  release- 
by-release  basis  using  concentrations  that  are  paired  in  space 
and  time  and  for  all  data/prediction  pairs  within  the  release. 
Additionally,  average  metrics  are  computed  by  averaging  each 
of  the  metrics  over  the  nine  releases. 
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STATISTICAL  COMPARISON  METRICS  EMPLOYED 


I  DA 


Fractional  Bias  (FB): 


Normalized  Absolute 
Difference  (NAD): 


Normalized  Mean 
Square  Error  (NMSE): 
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FACx  =  fraction  of  cases  for  which  l/x  <  Cp/C0<x ,  excluding 
prediction/  observation  pairs  for  which  Cp= 0  or  CQ= 0.  Specifically,  we 
employ  Fac2,  Fac5,  and  FaclO. 


All  metrics  are  computed  for  each  release  over  all  concentrations, 
paired  in  space  and  time  35 


AREA-BASED  MEASURE  OF  EFFECTIVENESS  (MOE): 
FALSE  NEGATIVES  AND  FALSE  POSITIVES 


The  figure  in  the  slide  shows  three  areas  used  in  formulating 
the  Area-based  Measure  of  Effectiveness  (MOE),  the  overlap 


area,  AOV,  the  false  negative  area,  AFN,  and  the  false  positive 
area,  AFP. 
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Area-Based  Measure  of  Effectiveness 

(MOE): 

False  Negatives  and  False  Positives 


Observation 


Afn  =  Area  of  False  Negative 

Aov=  Area  of  Overlap 

Afp  =  Area  of  False  Positive 

Aob  =  Area  of  Observation  =  Aov+AFN 
Apr  =  Area  of  Prediction  =  A0V+AFP 


37 


AREA-BASED  MEASURE  OF  EFFECTIVENESS  (MOE): 
FALSE  NEGATIVE  AND  FALSE  POSITIVE  FRACTIONS 


The  slide  shows  the  definition  of  the  false  negative  fraction 
f|'N  and  false  positive  fraction  fFP.  The  two-dimensional  MOE 


has  x  and  y  coordinates  given  respectively  by  MOEx  1-  fpN  and 
MOEy=l-  fFP. 
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Area-Based  Measure  of  Effectiveness  (MOE): 

False  Negative  and  False  Positive  Fractions 


False  Negative  Area  predicted  to  be  not  hazardous,  but  is  hazardous 

Fraction  Total  area  0f  observed  (true)  hazard 

For  a  uniform  population  density,  of  those  people  exposed,  this  is  the  fraction 
who  were  not  warned 
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False  Positive  Area  predicted  to  be  hazardous,  but  is  not  hazardous 
Fraction  Total  area  of  predicted  hazard 

For  a  uniform  population  density,  of  the  people  predicted  to  be  exposed,  this 
is  fraction  who  were  inadvertently  warned 

3 
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MOE  =  (MOEx,  MOEy)  =  (1-Ffn,  1-Ffp)  =  (Aov/Aob,  Aov/APR) 


The  figure  in  the  slide  shows  the  two-dimensional  MOE 
space,  which  is  the  unit  square.  Every  model 
prediction/observation  comparison  corresponds  to  a  point  in  this 
MOE  space.  A  “perfect”  MOE  of  (1,  1)  represents  complete 
agreement  between  predictions  and  observations,  while  a  MOE  of 
(0,  0)  represents  no  overlap  between  model  predictions  and 
observations.  From  the  definition  of  MOEx  and  MOEy  it  is 
straight-forward  to  show  that  the  diagonal  line  MOEy=MOEx 
corresponds  to  the  case  where  the  prediction  area  is  the  same  as 


the  truth  area,  APR=AOB.  Below  the  diagonal,  the  false 
positive  fraction  is  larger  than  the  false  negative  fraction  and  the 
model  is  said  to  overpredict  the  observations;  for  MOEx=l,  the 
model  predictions  completely  surround  the  observations  (no 
false  negative).  Above  the  diagonal,  the  false  positive  fraction 
is  smaller  than  the  false  negative  fraction  and  the  model  is  said 
to  underpredict  the  observations;  for  MOEy=l,  the  observations 
completely  surrounds  the  model  predictions  (no  false  positive). 
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SUMMED  CONCENTRATION  AND  THRESHOLD  MOES 


The  slide  gives  the  definition  of  two  kinds  of  MOES.  The 
summed  concentration  MOE  scores  the  model  based  on  the 
marginal  differences  between  the  predicted  and  observed 
concentrations.  The  threshold  MOE  scores  the  model  based  on 
the  number  of  samplers  that  are  predicted  and  observed  to  be 
above  a  given  concentration  threshold  T.  As  for  the  previous 


statistics,  all  MOEs  are  computed  on  a  release-by-release  basis 
using  concentrations  that  are  paired  in  space  and  time  and  for  all 
data/prediction  pairs  within  the  release.  A  more  detailed 
description  of  the  two-dimensional  MOE  can  be  found  in 
reference  [8]  and  references  therein. 
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SUMMED  CONCENTRATION  AND  THRESHOLD  MOES 
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Overlap  (Aov),  False  Negative  (AFN)  and  False 
Positive  (AFP)  “areas”  for  predictions  and 
observations  at  N  Samplers:  i=1,2,  N 
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All  MOEs  are  computed  for  each  release  over  all  concentrations, 
paired  in  space  and  time  43 


INDIVIDUAL  RELEASES:  GRAPHICAL  COMPARISONS 


The  nine  slides  that  follow  are  identical  in  format  and  show  =  0.6  m),  which  turned  out  to  yield  the  best  HP  AC -based 

graphical  comparisons  of  HP  AC  and  NARAC  predicted  1-hour  predictions  (as  is  shown  later  in  this  document).  The 

average  concentrations  and  observations.  The  HPAC  results  that  meteorology  option  was  “full  met”  in  all  cases, 
are  shown  are  for  Mode  1  (terrain  on,  constant  surface  roughness 
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Individual  Releases: 
Graphical  Comparisons 


9/20/78  1-HOUR  AVERAGES 


The  top  row  shows  the  predicted  HP  AC  (left)  and  NARAC 
(middle)  1-hour  average  concentrations  and  observations  (in 
ng/m3)  at  the  sampler  locations.  The  HP  AC  predictions  are  for 
Mode  1  (terrain  on,  constant  surface  roughness=0.6  m)  with  full 
met.  Only  2  sampler  arcs  (2  km  and  4  km)  were  employed  in  this 
release.  The  release  point  is  the  red  star  at  the  origin.  The  bar 
colors  mean  the  following.  The  height  of  a  green  bar  represents 
the  concentration  value  where  the  predicted  and  observed 
concentrations  agree  (overlap)  at  a  sampler  location.  A  red  bar 
represents  the  additional  concentration  observed,  but  not 

predicted  by  the  model  (a  false  negative).  A  yellow  bar 
represents  the  additional  concentration  predicted,  but  not 

observed  by  the  model  (a  false  positive). 

The  HP  AC  predictions  are  seen  to  underpredict  the 
observations  noticeably  more  than  NARAC  (HP AC  has  more 
false  negatives).  The  NARAC  plume  appears  to  be  shifted  to  the 
right  of  the  observations  when  viewed  from  the  release  location. 
This  would  account  for  the  excess  NARAC  overpredictions 
(yellow  bars)  relative  to  HPAC  (NARAC  has  more  false 
positives).  The  figure  on  the  top  right  shows  a  log-log  plot  of  the 
predicted  versus  observed  concentrations  for  HPAC  (blue  circles) 
and  NARAC  (red  squares).  The  dashed  diagonal  lines  are  factor 
of  two  lines  above  and  below  the  main  diagonal.  The  HPAC 


underpredictions  and  the  NARAC  overpredictions  are  clearly 
seen.  The  plot  in  the  bottom  left  shows  the  summed 
concentration  MOEs.  The  1-hour  average  concentration  MOEs 
are  represented  by  the  letter  H  -  blue  for  HPAC  and  red  for 
NARAC.  The  plot  shows,  quantitatively,  the  qualitative  result 
observed  from  the  sampler  plots,  namely  that  HPAC  exhibits  a 
higher  false  negative  fraction  than  NARAC,  while  NARAC 
exhibits  a  higher  false  positive  fraction  than  HPAC  for  this 
release.  Also  shown  are  the  HPAC  MOEs  for  the  3  20-minute 
average  concentration  measurement  periods  (blue  1,2,  and  3). 
The  plot  in  the  bottom  right  shows  the  threshold  MOEs  based 
on  a  threshold  of  20  ng/m3.  This  value  is  roughly  twice  the 
method  level  of  detection  (MLOD)  of  12  ng/m3  for  the  sampler 
measurement  devices  employed  in  the  Copenhagen  trials  [5].  It 
is  clear  from  this  plot  that  NARAC  and  HPAC  are  each  able  to 
predict  this  low  threshold  concentration  reasonably  well. 

The  table  in  the  bottom  center  shows  6  statistics  for  HPAC 
and  NARAC  for  this  release.  The  HPAC  underprediction  is 
evident  from  the  large  negative  fractional  bias,  FB  =  -0.77.  The 
NARAC  value  of  FB  =  0.005  is  a  reflection  of  compensating 
over  and  under  predictions  observed  in  the  sampler  plot.  Other 
statistics  indicate  better  NARAC  performance  for  this  release, 
for  example  the  smaller  NARAC  NAD  and  BNMSE  values. 
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9/20/78  1 -HOUR  AVERAGES 
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HPAC,  Terrain,  SR=0.6  m,  Full  Met  NARAC 


Predictions  vs  Observations 


9/20/76;  Hour  Average,  HPAC,  Terrain,  SR =0.6  m 


9/20/76;  Hour  Average,  NARAC 


1  Hour  Average  Predictions  vs  Observations: 


9/20/78 


Summed  Concentration  MOEs  Standard  Statistics 


20  ng/m3  Threshold  MOEs 


9/2fV70:  Summed  Cone.  MOEs;  HPAC  Mode:  Terrain,  SR -0,6  m  0/20/78:  20  ng'im3  Threshold  MOEs;  HPAC  Mode:  Terrain,  SR -0.6  m 
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9/26/78  1-HOUR  AVERAGES 


HP  AC  underpredictions  relative  to  NARAC  are  again  clear  observations.  The  qualitative  performance  of  NARAC  and 
from  the  sampler  concentration  figures.  The  NARAC  plume  HP  AC  for  this  release  is  the  same  as  was  found  in  the  9/20/78 

again  appears  to  be  shifted  to  the  right  with  respect  to  the  release. 
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Concentration  (ng/m3) 


9/26/78  1 -HOUR  AVERAGES 
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HPAC,  Terrain,  SR=0.6  m,  Full  Met  NARAC 


Predictions  vs  Observations 


9/26778:  Hour  Average,  HPAC,  Terrain,  SR- 0.6  m 


9/26/78:  Hour  Average,  NARAC 


Hour  Average  Predictions  vs  Observations; 
9/26/78 


Summed  Concentration  MOEs 


Standard  Statistics 


20  ng/m3  Threshold  MOEs 
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10/19/78  1-HOUR  AVERAGES 


Significant  HP  AC  underpredictions  are  seen  at  all  three  arcs.  NARAC  plume  shift,  as  in  the  two  previous  releases. 
Large  NARAC  overpredictions  are  consistent  with  a  right-going 
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10/19/78  1-HOUR  AVERAGES 
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HPAC,  Terrain,  SR=0.6  m,  Full  Met  NARAC 


Predictions  vs  Observations 


10/19/78:  Hour  Average,  HPAC,  Terrain,  SR- 0.6  m 


10/19/78:  Hour  Average,  NARAC 


1  Hour  Average  Predictions  vs  Observations: 
10/19/78 


Summed  Concentration  MOEs 

10/18/78:  Summed  Cone.  MOEs;  HPAC  Mode:  Terrain,  SR-0.6  m 


Standard  Statistics 


20  ng/m3  Threshold  MOEs 

10/19/70: 20  ng/m3  Threshold  MOEs;  HPAC  Mode:  Terrain,  SR-0.6  m 


MOE 


MQE 
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11/3/78  1-HOUR  AVERAGES 


HP  AC  underpredictions  are  higher  than  those  of  NARAC.  there  were  only  10  samplers  activated  for  this  release,  the  results 

The  HP  AC  false  positive  fraction  is  also  higher  than  that  of  may  not  represent  overall  model  performance  for  this  release. 

NARAC,  due  possibly  to  a  right-going  HPAC  plume  shift.  Since 
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11/3/78  1 -HOUR  AVERAGES 
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HP  AC,  Terrain,  SR=0.6  m,  Full  Met 

1 1/3/70:  Hour  Average.,  HP  AC,  Terrain,  SR =0.6  m 


NARAC 

1 1/3/78:  Hour  Average,  NARAC 


Predictions  vs  Observations 

1  Hour  Average  Predictions  vs  Observations: 
11/3/78 


Summed  Concentration  MOEs 

n/3/70:  Summed  Cone.  MOEs;  HP  AC  Mode:  Terrain,  SR  -0.6  m 


Standard  Statistics 


20  ng/m3  Threshold  MOEs 

1 1/3/70:  20  ng'm3  Threshold  MOEs;  HPAC  Mode:  Terrain,  SR-0.6  m 
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11/9/78  1-HOUR  AVERAGES 


Significant  HP  AC  false  negatives  are  observed  in  all  three  NARAC  plume  shift  with  respect  to  the  observations, 
arcs.  NARAC  false  positives  are  consistent  with  a  right-going 
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11/9/78  1 -HOUR  AVERAGES 
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HP  AC,  Terrain,  SR=0.6  m,  Full  Met 

1 1/9778:  Hour  Average,  HPAC,  Terrain,  SR-0.6  m 


NARAC 

1 1/9/78:  Hour  Average,  NARAC 


Predictions  vs  Observations 


1  Hour  Average  Predictions  vs  Observations: 
11/9/78 


Summed  Concentration  MOEs 

T 1/970:  Summed  Cone.  MOEs;  HP  AC  Mode:  Terrain,  SR -0,6  m 


Standard  Statistics 


20  ng/m3  Threshold  MOEs 

1 1/9/78:  20  ng'm3  Threshold  MOEs;  HPAC  Mode:  Terrain,  SR-0.6  m 
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4/30/79  1-HOUR  AVERAGES 


Both  HP  AC  and  NARAC  appear  to  have  significant  left-  concentration  MOE  plot  (lower  left),  which  shows  a  zero  false 

going  plume  shifts  with  respect  to  sampler  observations.  Every  positive  fraction.  NARAC  shows  significant  false  negatives,  as 

HP  AC  prediction  is  below  its  paired  observation,  as  evidenced  in  well, 
the  log-log  concentration  plot  (top  right)  and  the  summed 
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4/30/79  1 -HOUR  AVERAGES 
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HPAC,  Terrain,  SR=0.6  m,  Full  Met 


NARAC 


Predictions  vs  Observations 


4/30/79:  Hour  Average,  HPAC,  Terrain,  SR- 0.6  m 


4/30/79:  Hour  Average,  NARAC 


1  Hour  Average  Predictions  vs  Observations: 
4/30/79 


Summed  Concentration  MOEs  Standard  Statistics 


20  ng/m3  Threshold  MOEs 
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6/27/79  1-HOUR  AVERAGES 


Again,  left-going  plume  shifts  appear  to  be  present  in  the  larger  false  negative  fraction.  NARAC  has  the  larger  false 

HP  AC  and  NARAC  predictions.  Significant  underpredictions  positive  fraction. 

(false  negatives)  accompany  both  models,  with  HP  AC  having  the 
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Concentration  [ng/m3} 


6127179  1 -HOUR  AVERAGES 


I  DA 


HPAC,  Terrain,  SR=0.6  m,  Full  Met 


NARAC 


Predictions  vs  Observations 


6/27/79:  Hour  Average,  HPAC,  Terrain,  SR-&6  m  6/27/79:  Hour  Average,  NARAC 


1  Hour  Average  Predictions  vs  Observations: 
6/27/79 


Summed  Concentration  MOEs 

6/27/79:  Summed  Cone.  MOEs;  HPAC  Mode:  Terrain,,  SR -0,6  nn 


Standard  Statistics 


20  ng/m3  Threshold  MOEs 

6/27/79:  20  ng/m3  Threshold  MOEs;  HPAC  Mode:  Terrain,  SR -0.6  m 


59 


7/6/79  1-HOUR  AVERAGES 


For  this  release,  HP  AC  false  negatives  are  concentrated  in  the 
2  km  arc.  By  comparison,  NARAC  exhibits  very  few  false 
negatives;  those  present  are  also  primarily  in  the  2  km  arc. 
NARAC  has  the  larger  false  positive  fraction,  with  the  bulk  of  the 
false  positives  falling  in  the  4  km  and  6  km  arcs.  Interestingly, 


the  Fac2  performance  of  the  two  models  is  nearly  the  same  - 
0.600  for  HP  AC  and  0.641  for  NARAC,  yet  the  graphs  and 
summed  concentration  MOEs  show  substantially  different 
model  performance. 
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Concentration  [ng/m3} 


7/6/79  1 -HOUR  AVERAGES 


I  DA 


HPAC,  Terrain,  SR=0.6  m,  Full  Met  NARAC 


Predictions  vs  Observations 


7/6/79:  Hour  Average,  HPAC,  Terrain,  SR-G.6  m 


1  Hour  Average  Predictions  vs  Observations: 

7/6/79:  Hour  Average,  NARAC  7/6/79 


Summed  Concentration  MOEs 

7 /E/79:  Summed  Cone.  MOEs:  HPAC  Mode:  Terrain,  SR-D.G  m 


Standard  Statistics 


20  ng/m3  Threshold  MOEs 

7/6/79: 20  ng/m3  Threshold  MOEs;  HPAC  Mode:  Terrain,  SR-G.B  m 


MOE 


MOE 
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7/19/79  1-HOUR  AVERAGES 


The  HP  AC  plume  appears  to  have  a  left-going  shift,  as  in  the  the  2  km  arc.  NARAC  has  the  higher  false  positive  fraction  and 

4/30/79  and  6/27/79  releases.  Large  HP  AC  false  negatives  are  the  smaller  false  negative  fraction, 

observed  on  all  arcs.  NARAC  false  negatives  are  prominent  in 
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7/19/79  1-HOUR  AVERAGES 


I  DA 


HPAC,  Terrain,  SR=0.6  m,  Full  Met 

7/1 S/79;  Hour  Average,  HPAC  Terrain,  SR =0.6  m 
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NARAC 

7/1 9/79:  Hour  Average,  NARAC 


-2 

North  (km) 


2 

East  (km) 


Predictions  vs  Observations 

1  Hour  Average  Predictions  vs  Observations: 
7/19/79 


Summed  Concentration  MOEs 

7/6/79:  Summed  Cone.  MOEs:  HPAC  Mode:  Terrain,  SFV0.G  m 


Standard  Statistics 


20  ng/m3  Threshold  MOEs 

7/19/79:  20  ng/m3  Threshold  MOEs;  HPAC  Mode:  Terrain,  SR -0.6  m 


MOE 


MOE 
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OVERALL  NARAC  AND  HPAC  PERFORMANCE: 
STATISTICAL  COMPARISONS 


The  following  several  slides  focus  on  the  overall  performance 
of  HPAC  and  NARAC. 


Graphical  and  statistical  comparisons  of  the  two  models  are 
shown. 
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Overall  NARAC  and  HPAC  Performance: 

Statistical  Comparisons 
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HP  AC  &  NARAC  NORMALIZED  ABSOLUTE  DIFFERENCE: 

1  HOUR  AVERAGE  CONCENTRATIONS;  FOUR  HP  AC  MODES,  FULL  MET 

The  graph  shows  the  normalized  absolute  difference  (NAD)  NAD  values  for  all  4  HP  AC  modes.  The  smallest  HPAC  NAD 
performance  of  NARAC  and  all  four  HP  AC  modes  using  full  met  value  for  every  release  is  the  one  performed  with  constant 
for  all  releases.  With  the  exception  of  the  11/9/79  release,  surface  roughness;  of  those,  the  majority  are  HPAC  Mode  1: 
NARAC  NAD  values  (red  filled  squares)  are  smaller  than  HPAC  terrain  on,  surface  roughness  =  0.6  m  (blue  filled  diamonds). 
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HPAC  &  NARAC  Normalized  Absolute  Difference: 

1  Hour  Average  Concentrations;  Four  HPAC  Modes,  Full  Met 

Normalized  Absolute  Difference:  1  Hour  Averages 
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•  With  the  exception  of  the  11/9/79  release,  NARAC  NAD  values  are  smaller  than 
HPAC  NAD  values  for  all  4  HPAC  modes 

•  The  smallest  HPAC  NAD  value  for  every  release  is  the  one  performed  with 
constant  surface  roughness;  of  those,  the  majority  are  HPAC  Mode  1:  Terrain  on, 
SR=0.6m 


Normalized  Absolute  Difference:  1  Hour  Averages 
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HP  AC  &  NARAC  FRACTIONAL  BIAS:  1  HOUR  AVERAGE 
CONCENTRATIONS;  FOUR  HPAC  MODES,  FULL  MET 


The  graph  shows  the  fractional  bias  (FB)  performance  of 
NARAC  and  all  four  HPAC  modes  using  full  met  for  all  releases. 
Every  HPAC  mode  underpredicts  every  release  (FB<0),  while 
NARAC  underpredicts  7  of  the  9  releases  (the  two  exceptions  are 
the  9/20/78  and  7/6/79  releases).  For  every  release,  all  HPAC 
modes  underpredict  NARAC  (i.e.,  all  HPAC  modes  have  more 


negative  FBs).  The  least  negative  HPAC  FB  value  for  every 
release  is  one  performed  with  constant  surface  roughness;  of 
those,  the  majority  are  HPAC  Mode  1  (terrain  on,  surface 
roughness  =  0.6  m)  (blue  filled  diamonds).  This  result  is 
consistent  with  the  NAD  result  in  the  previous  slide. 
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HPAC  &  NARAC  Fractional  Bias:  1  Hour  Average 
Concentrations;  Four  HPAC  Modes,  Full  Met 

0.4 
0.2 
0.0 
-0.2 
-0.4 
£-0  6 
-0.8 
-1.0 
-1.2 
-1.4 
-1.6 


Fractional  Bias:  1  Hour  Averages 
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•  HPAC  underpredicts  (FB  <  0)  every  release  for  all  4  modes 

•  NARAC  underpredicts  7  of  the  9  releases  (exceptions  are  9/20/78  and  7/6/79) 

FB  values  for  all  HPAC  modes  are  more  negative  than  NARAC  for  every  release 

•  The  least  negative  HPAC  FB  value  for  every  release  is  the  one  performed  with  constant 
surface  roughness;  of  those,  the  majority  are  HPAC  Mode  1:  Terrain  on,  SR=0.6m 
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HPAC  AND  NARAC  AVERAGE  STATISTICS:  1  HOUR  AVERAGE 
CONCENTRATIONS;  FOUR  HPAC  MODES,  FULL  MET 

The  table  in  the  slide  shows  the  average  statistics  of  NARAC  of  <FaclO>,  HPAC  Mode  1  (terrain  on,  surface  roughness  =  0.6 

and  the  four  HPAC  modes  with  full  met.  With  the  exception  of  m)  average  statistics  outperform  the  average  statistics  for  all 

<FaclO>,  NARAC  average  statistics  (red)  outperform  HPAC  other  HPAC  modes, 

average  statistics  for  all  HPAC  modes.  Again,  with  the  exception 
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HPAC  and  NARAC  Average  Statistics:  1  Hour  Average 
Concentrations;  Four  HPAC  Modes,  Full  Met 


Average  Statistics  for  4  HPAC  Modes  (Full  met)  and  NARAC 


<FB> 

<NAD> 

<NMSE> 

<BNMSE> 

<FAC2> 

<FAC5> 

<FAC10> 

HPAC  Mode  1: 
Terrain,  SR=0.6 

-0.82 

0.50 

3.65 

0.31 

0.31 

0.73 

0.90 

HPAC  Mode  2: 

Terrain,  Landcover 

-0.99 

0.57 

5.24 

0.40 

0.27 

0.70 

0.91 

HPAC  Mode  3:  No 
terrain,  SR=0.6 

-0.85 

0.53 

3.85 

0.34 

0.30 

0.68 

0.86 

HPAC  Mode  4:  No 
terrain,  Landcover 

-0.99 

0.58 

5.28 

0.41 

0.26 

0.69 

0.90 

NARAC 

-0.20 

0.37 

1.31 

0.15 

0.44 

0.74 

0.84 

•  With  the  exception  of  <FAC10>,  NARAC  average  statistics 
outperform  HPAC  average  statistics  for  all  HPAC  modes. 

•  With  the  exception  of  <FAC10>,  HPAC  mode  1:  Terrain  on,  SR=0.6  m 
average  statistics  outperform  average  statistics  for  all  other  HPAC 
modes. 
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HP  AC  MODE  1,  FULL  MET  AND  NARAC: 
ALL  RELEASES  -  1  HOUR  AVERAGES 


The  plot  in  the  top  left  shows  a  log-log  plot  of  the  HPAC 
(Mode  1,  full  met)  and  NARAC  predictions  versus  observations 
in  ng/m3  for  all  releases.  The  plot  on  the  right  shows  the 
normalized  histograms  of  HPAC  and  NARAC  predictions  and 
observations.  For  low  concentrations  (~  50  ng/m3  and  below)  the 
three  histograms  are  similar.  Between  -50-500  ng/m3  HPAC 
(blue)  overpredicts  both  the  NARAC  predictions  (red)  and  the 
observations  (green). 


At  around  1000  ng/m3  there  is  a  peak  in  the  NARAC 
histogram  and  a  noticeable  dip  in  the  observation  histogram.  At 
~  2000  ng/m3  and  above,  the  NARAC  and  observation 
histograms  track  each  other  through  ~  4000  ng/m3.  The  table 
shows  summary  statistics  for  HPAC  and  NARAC.  The 
statistics  show  the  significant  underpredictive  tendency  of 
HPAC  and  the  less  pronounced  underpredictive  tendency  of 
NARAC  for  these  releases.  The  bottom  two  rows  indicate  the 
fraction  of  cases  for  which  NARAC  overpredicted  HPAC 
(HPAC  overpredicted  NARAC). 
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1  Hour  Average  Predictions  vs  Observations 
All  Releases 


HP  AC  Mode  1,  Full  Met  and  NARAC: 

All  Releases  - 1  Hour  Averages 

Histogram  Comparison:  1  Hour  Average  Concentrations 
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1  Hour  Average  Concentration  (ng/m  ) 


NARAC 

HPAC 

OBSERVATIONS 

Mean  (ng/mJ) 

690 

347 

846 

Median  (ng/mJ) 

394 

210 

399 

Fac2 

0.419 

0.332 

Fac5 

0.740 

0.732 

FacIO 

0.834 

0.883 

Fraction  underpredicted 

0.570 

0.774 

Fraction  overpredicted 

0.430 

0.226 

Fraction  NARAC  >  HPAC 

0.728 

Fraction  HPAC  >  NARAC 

0.272 
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10,000  RESAMPLED  MOES;  HP  AC  MODE  1,  FULL  MET  AND  NARAC: 
ALL  RELEASES  -  1  HOUR  AVERAGES 


The  top  figure  shows  10,000  resampled  HP  AC  and  NARAC 
summed  concentration  MOEs  for  the  1-hour  average 
concentrations  for  the  9  releases.  The  HP  AC  and  NARAC  clouds 
are  clearly  separated,  with  HP  AC  exhibiting  more  false  negatives 
and  fewer  false  positives  than  NARAC.  The  bottom  figure  shows 
10,000  resampled  HP  AC  and  NARAC  threshold  MOEs  based  on 
a  20  ng/m3  threshold.  There  is  significant  overlap  between  the 


HPAC  and  NARAC  clouds,  indicating  similar  low 
concentration  (e.g.,  hazard  area  prediction)  model  performance. 
Since  the  detection  limit  is  approximately  10  ng/m3  ,  the 
instrument  noise  is  high  for  this  low  concentration  threshold. 
There  are  noticeable  differences  between  the  two  models’ 
relative  performance  when  examining  thresholds  well  above  (> 
lOx)  the  instrument  detection  limit. 
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10,000  Resampled  MOEs;  HPAC  Mode  1,  Full  Met  and  NARAC: 

All  Releases  - 1  Hour  Averages 


Resampled  Summed  Concentration  MOEs: 
All  Releases.  1  Hour  Averages 


The  Summed  Concentration  MOE  tests  a  model’s  ability 
to  predict  the  amount  gf_matena[  at  the  samplers 

•HPAC  exhibits  sizeable  false  negatives  and  few  false  positives 
•NARAC  exhibits  fewer  false  negatives  and  somewhat  more 
false  positives 

•The  performance  of  HPAC  and  NARAC  is  clearly  separated 


0.0  0.2  0.4  0.6  0.8  1.0 

MOEx 


The  Threshold  MOE  tests  a  model’s  ability  to  predict  the  number 
of  samplers  that  exceed  the  threshold ,  in  this  case  a  low  value 
of  20  ng/m3  (~  2  times  minimum  detectable  concentration) 

•False  negatives  and  false  positives  for  HPAC  and  NARAC  are 
greatly  reduced  with  respect  to  Summed  Concentration  MOE 
•HPAC  and  NARAC  show  very  similar  performance 
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Resampled  20  ng/m3  Threshold  MOEs: 
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HP  AC  AND  NARAC  SUMMED  CONCENTRATION  MOE  DIFFERENCES  (NARAC- 
HPAC):  1  HOUR  AVERAGE  CONCENTRATIONS 


The  figure  on  the  left  shows  the  summed  concentration  MOE 
differences  between  NARAC  and  HP  AC  (NARAC  MOE  minus 
HP  AC  MOE).  Two  of  the  nine  differences  lie  in  the  (+,+) 
quadrant,  showing  fewer  false  negatives  and  fewer  false  positives 


for  NARAC.  The  remaining  differences  lie  in  the  (+,-) 
quadrant,  showing  fewer  false  negatives  and  more  false 
positives  for  NARAC.  Sampler  plots  for  the  two  (+,  +) 
differences  are  also  shown. 
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HPAC  and  NARAC  Summed  Concentration  MOE  Differences 
(NARAC-HPAC):  1  Hour  Average  Concentrations 
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MODE  1  HP  AC  VS  NARAC  STATISTICAL  DIFFERENCES: 
1  HOUR  AVERAGE  CONCENTRATIONS 


The  slide  shows  the  results  of  applying  statistical  hypothesis 
testing  to  the  HPAC  (Mode  1,  full  met)  and  NARAC  statistics  for 
the  nine  releases.  The  null  hypothesis  is  that  NARAC  and  HPAC 
do  not  differ.  The  nonparametric  statistical  test  that  is  applied  is 
a  randomization  test  (the  permutation  test)  on  the  paired 
differences  between  HPAC  and  NARAC  statistics.  A  description 
of  this  technique  with  an  application  to  urban  T&D  field  trials 
can  be  found  in  reference  [9]  and  references  therein.  Under  the 
null  hypothesis,  the  observed  differences  for  each  release  are 
equally  likely  to  be  positive  or  negative.  The  test  generates  all 


29=5 12  permutations  of  the  9  paired  differences,  computes  their 
mean  values,  orders  the  absolute  mean  differences,  and 
computes  the  probability  (the  p-value)  of  matching  or  exceeding 
that  value.  The  p-value  for  the  observed  absolute  mean 
difference  is  then  obtained.  Under  this  test  the  FB,  NAD, 
NMSE,  and  BNMSE  differences  are  all  statistically  significant 
at  the  2%  level  (p-value  <  0.02;  see  green  values  in  the  table). 
That  is,  for  these  statistics  we  reject  the  null  hypothesis  that 
HPAC  and  NARAC  are  the  same  at  the  98  percent  level. 
Similar  results  were  found  for  the  3  other  HPAC  modes  with 
full  met. 
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Mode  1  HPAC  vs  NARAC  Statistical  Differences: 
1  Hour  Average  Concentrations 


•  Differences  in  HPAC  and  NARAC  statistics  were  examined  for  statistical  significance 

-  Null  hypothesis:  HPAC  and  NARAC  do  not  differ 

-  Applied  randomization  test  on  paired  differences  (aka  permutation  test)  and  compute  mean 
values  over  all  permutations  (standard  nonparametric  statistical  test) 

-  For  these  small  data  sets  (9  paired  differences ),  there  are  29  =  512  possible  means  => 
p-values  can  be  computed  exactly 

•  Findings 

-  For  HPAC  Mode  1  (Full  met),  FB,  NAD,  NMSE,  and  BNMSE  differences  are  all  statistically 
significant  at  2%  level  (p^ralue  <  0.02).  e.g.,  green  values  in  table  below 

-  Similar  findings  for  3  other  HPAC  modes  (Full  met) 


1  Hour  Avg.  Cone.  Standard  Statistics:  Terrain,  Surface  Roughness  =  0.6  m 

FB 

NAD 

NMSE 

BNMSE 

FAC  2 

FAC  5 

FAC  10 

HPAC 

NARAC 

HPAC 

NARAC 

HPAC 

NARAC 

HPAC 

NARAC 

HPAC 

NARAC 

HPAC 

NARAC 

HPAC 

NARAC 

9/20/78 

-0.766 

0.005 

0.430 

0.320 

2.536 

0.753 

0.234 

0.089 

0.387 

0.387 

0.903 

0.742 

0.968 

0.774 

9/26/78 

-0.814 

-0.236 

0.461 

0.418 

2.847 

1.400 

0.296 

0.184 

0.458 

0.417 

0.708 

0.667 

0.875 

0.875 

10/19/78 

-0.873 

-0.184 

0.607 

0.508 

4.140 

1.838 

0.373 

0.213 

0.118 

0.182 

0.441 

0.515 

0.618 

0.667 

11/3/78 

-0.687 

-0.220 

0.464 

0.197 

1.941 

0.344 

0.260 

0.049 

0.250 

0.875 

1.000 

0.875 

1.000 

1.000 

11/9/78 

-0.557 

-0.083 

0.462 

0.554 

1.779 

2.029 

0.194 

0.250 

0.321 

0.208 

0.643 

0.417 

0.893 

0.458 

4/30/79 

-1.402 

-0.660 

0.703 

0.381 

8.654 

1.439 

0.518 

0.154 

0.048 

0.286 

0.429 

0.905 

0.952 

1.000 

6/27/79 

-0.945 

-0.400 

0.536 

0.464 

5.542 

2.639 

0.424 

0.276 

0.370 

0.348 

0.717 

0.652 

0.826 

0.826 

7/6/79 

-0.551 

0.239 

0.351 

0.271 

2.527 

0.809 

0.230 

0.075 

0.600 

0.641 

0.875 

1.000 

0.975 

1.000 

7/19/79 

-0.781 

-0.261 

0.469 

0.216 

2.848 

0.573 

0.248 

0.058 

0.256 

0.615 

0.872 

0.897 

0.974 

0.949 

Mean  Value 

-0.820 

-0.200 

0.498 

0.370 

3.646 

1.314 

0.309 

0.150 

0.312 

0.440 

0.732 

0.741 

0.898 

0.839 

p-value 

0.0039 

0.0195 

0.0078 

0.0078 

0.1719 

0.9297 

0.4688 
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MULTIVARIATE  ANALYSIS  OF  VARIANCE  (MANOVA)  APPLIED  TO  METRICS: 

1  HOUR  AVERAGE  CONCENTRATIONS 


Another  method  to  analyze  the  statistical  metrics  is  to  apply  An  F  statistic  is  applied  to  a  function  of  the  largest 

multivariate  analysis  of  variance  (MANOVA)  [10]  to  several  eigenvalue  to  estimate  significance  (note  that  unlike  the 

metrics  simultaneously.  The  method  involves  construction  of  two  previous  test,  this  test  is  a  parametric  statistical  test).  The  test 

matrices  H  and  E  (reviewed  in  the  slide)  and  computing  the  was  applied  to  the  metrics  FB,  NAD,  BNMSE,  FAC2,  FAC5, 

largest  eigenvalue  of  the  matrix  P=HE_I.  and  FAC  10.  The  H  and  E  matrices  are  shown  along  with  the 

largest  eigenvalue,  F  statistic,  and  significance.  Under  this  test 
the  HP  AC  and  NARAC  differences  are  significant  at  a  level  of 
[10]  Morrison,  Donald  F.,  Multivariate  Statistical  Methods,  3rd  less  than  1%. 

Ed.,  McGraw-Hill,  1990. 
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Multivariate  Analysis  of  Variance  (MANOVA)  Applied  to  Metrics: 

1  Hour  Average  Concentrations 


•  Multivariate  Analysis  of  Variance  (MANOVA):  Method  to  estimate  the  significance  of  several 
metrics  taken  together 

-  Two  matrices  formed  (c.f.,  Morrison,  Donald  F.,  Multivariate  Statistical  Methods,  3rd  Ed.,  McGraw-Hill,  1990, 

p.  210) 

•  H:  “Hypothesis  sum  of  squares  matrix” 

•  E:  “Error  sum  of  squares  matrix” 

-  Form  the  matrix  product  P=HE_1 

-  Compute  largest  eigenvalue  of  P 

-  Significance  is  computed  from  F  statistic  applied  to  a  function  of  the  largest  eigenvalue  (note:  this  is  a 
parametric  statistical  test) 


•  Results 

-  Multivariate  ANOVA  applied  to:  FB,  NAD,  BNMSE,  FAC2,  FAC5,  and  FAC10 

-  SPSS  used  to  perform  analysis* 


H  Matrix 

FB 

NAD 

BNMSE 

FAC2 

FAC5 

FAC  10 

E  Matrix 

FB 

NAD 

BNMSE 

FAC2 

FAC5 

FAC  10 

FB 

1.727 

-0.357 

-0.443 

0.357 

0.025 

-0.165 

FB 

1.027 

-0.231 

-0.257 

0.348 

0.264 

-0.041 

NAD 

-0.357 

0.074 

0.092 

-0.074 

-0.005 

0.034 

NAD 

-0.231 

0.216 

0.162 

-0.332 

-0.31 

-0.192 

BNMSE 

-0.443 

0.092 

0.113 

-0.091 

-0.007 

0.042 

BNMSE 

-0.257 

0.162 

0.151 

-0.21 

-0.226 

-0.118 

FAC2 

0.357 

-0.074 

-0.091 

0.074 

0.005 

-0.034 

FAC2 

0.348 

-0.332 

-0.21 

0.644 

0.417 

0.283 

FAC5 

0.025 

-0.005 

-0.007 

0.005 

0 

-0.002 

FAC5 

0.264 

-0.31 

-0.226 

0.417 

0.632 

0.396 

FAC  10 

-0.165 

0.034 

0.042 

-0.034 

-0.002 

0.016 

FAC  10 

-0.041 

-0.192 

-0.118 

0.283 

0.396 

0.384 

-  Largest  eigenvalue  of  P=HE_1:  X=3.22Z 

-  F  statistic:  F=5.909 

-  Significance:  p=0.003  =>  Reject  null  hypothesis  that  HPAC  and  NARAC  results  are  the  same  at  a  significance 
level  of  less  than  1  % 


*Reviewer  Dr.  Dennis  DeRiggi  provided  this  analysis 
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HP  AC  “OPERATIONAL”  METEOROLOGY  PERFORMANCE 


The  graph  in  the  slide  shows  the  FB  performance  of  HP  AC 
Mode  1  with  the  three  meteorology  options,  Full  met  (blue  filled 
diamonds),  Operational  (black  stars),  and  Operational  with 
Temperature  (green  circles),  along  with  the  NARAC  FB  values 
(red  filled  squares).  With  the  exception  of  the  11/3/79  release, 
Operational  and  Operational  with  Temperature  HP  AC  predictions 
performed  better  than  or  as  well  as  Full  met  predictions;  NAD, 


NMSE,  and  BNMSE  show  the  identical  trend.  We  note  that  the 
1 1/3/79  release  has  a  boundary  layer  height  of  390  m,  which  is 
the  lowest  reported  boundary  layer  height  of  all  of  the 
Copenhagen  releases.  The  graph  shows  that  adding  temperature 
to  the  operational-only  met  did  not  improve  the  predictions.  At 
this  point  it  is  not  clear  why  the  Operational  meteorology 
outperforms  the  Full  meteorology  for  these  releases. 
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^  HPAC  “Operational”  Meteorology  Performance 
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•  With  the  exception  of  the  11/3/78  release ,  “Operational”  and  “Operational  with 
Temperature”  HPAC  predictions  performed  better  than  or  as  well  as  “Full  Met” 
predictions  (other  measures  show  identical  trend).  Note  that  11/3/78  release  has 
390  m  reported  BL  height  of  all  release  (next  lowest  is  810  m). 

•  Including  temperature  in  Operational  Met  did  not  improve  predictions  noticeably 

•  Not  clear  at  this  point  why  Operational  met  outperforms  Full  met 
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HPAC  VERTICAL  CONCENTRATION  SLICES: 
1000  NG/M3,  7/6/79  RELEASE 


The  figure  shows  HPAC  vertical  concentration  slices  at  1000 
ng/m3  roughly  down  the  plume  centerline  for  several  times  after 
the  start  of  the  7/6/79  release  (for  reference,  the  peak  and  mean 
observed  concentrations  on  the  first  arc  for  this  release  are  2928 
ng/m3  and  965  ng/m3  respectively).  The  1  minute  contour 
impacts  the  ground  and  extends  to  almost  three  times  the  release 
height,  indicating  a  large  vertical  diffusivity.  After  85  minutes 


(the  length  of  the  release)  the  downwind  extent  of  the  1000 
ng/m3  contour  at  ground  level  does  reach  the  first  sampler  arc. 
Similar  behavior  is  observed  in  the  HPAC  vertical  slice 
concentration  histories  for  the  other  releases.  It  is  reasonable  to 
assume  the  unaccounted  for  mass  in  the  HPAC  simulations 
remains  lofted  or  impacts  the  ground  before  the  sampler  grid.  A 
large  modeled  vertical  diffusivity  relative  to  actual  atmospheric 
conditions  is  one  possible  explanation  for  this. 
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HPAC  Vertical  Concentration  Slices: 
1000  ng/m3,  7/6/79  Release 


First  sampler  arc  observations 

Peak  concentration  =  2928  ng/m3 
Mean  concentration  =  965  ng/m3 

After  1  minute,  the  HPAC  1000  ng/m3 
contour  impacts  the  ground  and  extends 
to  ~  3  times  the  release  height 


400 


7/6/79  :  HPAC  Vertical  Concentration  Slices:  1000  ng/m3 


At  85  minutes  (release  duration)  the 
1000  ng/m3  contour  at  ground  level 
does  not  extend  to  the  first  sampler  arc 
(2  km  downwind) 


85  min 


Downwind  (km) 


Reasonable  to  assume  that  unaccounted  for  mass  in  HPAC  simulations 
is  lofted  or  already  on  the  ground  before  the  sampler  arcs  -  large 
modeled  vertical  diffusivity  relative  to  atmospheric  conditions  is  a 
possible  explanation  for  this 


Similar  behavior  is  observed  in  the  HPAC  vertical  slice  concentration 
histories  for  the  other  releases 
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MAJOR  FINDINGS 


The  slide  highlights  the  major  findings  of  this  study.  The 
main  finding  is  that  both  HP  AC  and  NARAC  underpredict  the  1- 
hour  average  ground  concentrations  of  the  elevated  Copenhagen 
releases,  with  the  HP  AC  underpredictions  being  much  more 
pronounced  than  those  of  NARAC.  With  respect  to  the  statistics 
FB,  NAD,  NMSE,  and  BNMSE,  the  differences  between  HP  AC 
and  NARAC  are  statistically  significant  at  the  2  percent  level. 
Multivariate  ANOVA  hypothesis  applied  to  FB,  NAD,  BNMSE, 


Fac2,  Fac5,  and  FaclO  shows  that  HP  AC  and  NARAC  differ  at 
the  1%  level  of  significance.  Resampled  HP  AC  and  NARAC 
summed  concentration  MOEs  are  also  well  separated.  The  4 
HP  AC  terrain  /landcover  modes  and  the  3  met  options  do  not 
“fix”  the  severe  HPAC  underpredictions.  Finally,  the  HPAC 
vertical  slice  concentration  contours  suggest  that  the  modeled 
HPAC  vertical  diffusivity  is  larger  than  what  was  present  during 
the  releases. 
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MAJOR  FINDINGS 


•  HPAC  and  NARAC  tend  to  underpredict  these  elevated  releases  at  the 
sampler  locations,  with  HPAC  underpredictions  being  more 
pronounced 


Ratio 

NARAC 

HPAC 

<  Obs  >  /  <  Pred  > 

1.22 

2.44 

med(  Obs)  /med(  Pred) 

1.01 

1.90 

•  HPAC  and  NARAC  differences  are  statistically  significant  at  the  2% 
level  with  respect  to  FB,  NAD,  NMSE,  and  BNMSE  (under  MANOVA, 
HPAC  and  NARAC  differences  are  significant  at  the  1  %  level) 

•  Resampled  (0.99  confidence  regions)  HPAC  and  NARAC  summed 
concentration  MOEs  are  completely  separated  (implying  statistically 
significant  differences) 

•  HPAC  Mode  and  Met  options  do  not  “fix”  the  HPAC  underpredictions 

•  HPAC  vertical  concentration  slice  contours  suggest  modeled  vertical 
diffusivity  is  larger  than  atmospheric  conditions  during  the  releases 


87 


SECONDARY  FINDINGS 


The  secondary  findings  of  this  study  are  listed  in  the  slide. 
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SECONDARY  FINDINGS 


•  At  at  low  threshold  (20  ng/m3  -  2  times  minimum  detectable  concentration), 
the  MOE  performance  of  HPAC  and  NARAC  are  both  favorable  and  nearly  the 
same  (slight  false  positive) 

•  At  thresholds  well  above  the  instrument  noise  (>  lOx  instrument  limit), 
differences  in  the  models  relative  and  absolute  accuracy  are  more  evident 

•  HPAC  predictions  conducted  with  constant  surface  roughness  outperformed 
those  conducted  with  landcover  enabled;  In  general,  “Terrain  on”,  SR=0.6  m 
performed  the  best 

-  Nonintuitive  result:  perhaps  current  landcover  database  does  not  represent  the 
1978-1979  Copenhagen  area  landcover 

•  Operational  weather  profiles  (wind,  speed  and  direction  only  or  with 
temperature)  yielded  better  HPAC  predictions  than  those  with  full 
meteorological  inputs,  except  for  one  of  the  9  releases  -  not  clear  why  this  is 
happening. 

-  Further  investigation  is  required  to  understand  this  result 
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BACKUPS 
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North  (km)  North  (km)  North  (km) 


ACTIVE  SAMPLERS 


ID  A 


9/20/78  Active  Sampler  Locations 


East  (km) 

11/3/78  Active  Sampler  Locations 


East  (km) 

6/27/79  Active  Sampler  Locations 


9/26/78  Active  Sampler  Locations 


East  (km) 

1 1/9/78  Active  Sampler  Locations 


East  (km) 

7/6/79  Active  Sampler  Locations 


East  (km) 
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10/19/78  Active  Sampler  Locations 


East  (km) 

4/30/79  Active  Sampler  Locations 


East  (km) 

7/19/79  Active  Sampler  Locations 


East  (km) 


East  (km) 


ALL  SAMPLER  LOCATIONS  -  UTM  ZONE  33 

Arc  1  Arc  2  Arc  3 


Sampler 

Number 

x  (m) 

y  (m) 

UTM-East 

(m) 

UTM-North 

(m) 

Arcl-l 

866 

-1545 

343387 

6178033 

Arcl-2 

952 

-1546 

343473 

6178029 

Arcl-3 

1038 

-1532 

343559 

6178040 

Arcl-4 

1141 

-1563 

343661 

6178005 

Arcl-5 

1233 

-1586 

343752 

6177978 

Arcl-6 

1323 

-1558 

343843 

6178003 

Arcl-7 

1345 

-1486 

343868 

6178074 

Arcl-8 

1371 

-1403 

343897 

6178156 

Arcl-9 

1398 

-1312 

343927 

6178246 

Arcl-10 

1404 

-1214 

343937 

6178343 

Arcl-11 

1492 

-1131 

344028 

6178423 

Arcl-12 

1516 

-1044 

344055 

6178509 

Arcl-13 

1582 

-964 

344124 

6178587 

Arcl-14 

1592 

-884 

344137 

6178666 

Arcl-15 

1602 

-798 

344150 

6178752 

Arcl-16 

1703 

-767 

344253 

6178779 

Arcl-17 

1766 

-681 

344319 

6178862 

Arcl-18 

1800 

-593 

344356 

6178949 

Arcl-19 

1877 

-485 

344437 

6179054 

Arcl-20 

1921 

-405 

344484 

6179132 

Arcl-21 

2067 

-371 

344631 

6179161 

Arcl-22 

2061 

-284 

344629 

6179248 

Arcl-23 

2055 

-180 

344627 

6179352 

Arc  1-24 

2050 

-82 

344625 

6179450 

Arcl-25 

2044 

20 

344623 

6179552 

Arcl-26 

2038 

119 

344621 

6179651 

Arcl-27 

2032 

225 

344619 

6179758 

Arcl-28 

2026 

329 

344617 

6179862 

Arcl-29 

2000 

434 

344595 

6179968 

Arcl-30 

1978 

524 

344577 

6180058 

Arcl-31 

2008 

626 

344610 

6180159 

Arcl-32 

2002 

758 

344609 

6180291 

Arcl-33 

1996 

854 

344607 

6180387 

Arcl-34 

1991 

956 

344606 

6180490 

Arcl-35 

1995 

1081 

344615 

6180614 

Arcl-36 

1986 

1187 

344610 

6180721 

Arcl-37 

1969 

1287 

344597 

6180821 

Arcl-38 

1880 

1333 

344509 

6180871 

Arcl-39 

1829 

1407 

344461 

6180946 

Arcl-40 

1750 

1480 

344385 

6181022 

Arcl-41 

1697 

1566 

344335 

6181110 

Arcl-42 

1594 

1657 

344236 

6181205 

Arcl-43 

1507 

1708 

344151 

6181259 

Arcl-44 

1440 

1742 

344085 

6181296 

Arcl-17A 

2094 

-761 

344644 

6178770 

Arcl-18A 

2087 

-638 

344641 

6178893 

Arcl-19A 

2078 

-538 

344636 

6178993 

Arcl-20A 

2071 

-446 

344633 

6179086 

Sampler 

Number 

x  (m) 

y  (m) 

UTM-East 

(m) 

UTM-North 

(m) 

Arc3-2 

2898 

-4900 

345290 

6174603 

Arc3-3 

3002 

-4842 

345396 

6174657 

Arc3-4 

3139 

-4614 

345541 

6174880 

Arc3-5 

3342 

-4276 

345757 

6175210 

Arc3-6 

3541 

-4118 

345962 

6175360 

Arc3-7 

3693 

-3919 

346121 

6175553 

Arc3-8 

3883 

-3752 

346317 

6175713 

Arc3-9 

4027 

-3616 

346467 

6175843 

Arc3-10 

4283 

-3447 

346729 

6176003 

Arc3-ll 

4390 

-3277 

346842 

6176168 

Arc3-12 

4459 

-3010 

346921 

6176433 

Arc3-13 

4572 

-2795 

347042 

6176643 

Arc3-14 

4668 

-2514 

347149 

6176920 

Arc3-15 

4824 

-2260 

347315 

6177168 

Arc3-16 

5029 

-2108 

347525 

6177312 

Arc3-17 

5286 

-1939 

347788 

6177471 

Arc3-18 

5378 

-1570 

347894 

6177837 

Arc3-19 

5395 

-1399 

347918 

6178007 

Arc3-20 

5375 

-1139 

347908 

6178267 

Arc3-21 

5323 

-913 

347864 

6178495 

Arc3-22 

5305 

-690 

347855 

6178719 

Arc3-23 

5314 

-456 

347873 

6178952 

Arc3-24 

5264 

-218 

347832 

6179192 

Arc3-25 

5229 

35 

347807 

6179446 

Arc3-26 

5268 

277 

347855 

6179686 

Arc3-27 

5354 

558 

347951 

6179964 

Arc3-28 

5618 

914 

348229 

6180310 

Arc3-29 

5829 

1208 

348451 

6180595 

Arc3-30 

5952 

1544 

348586 

6180926 

Arc3-31 

5608 

1827 

348253 

6181222 

Arc3-32 

5809 

2176 

348468 

6181563 

Arc3-33 

5451 

2381 

348118 

6181782 

Arc3-34 

5259 

2601 

347934 

6182009 

Arc3-35 

5213 

2880 

347899 

6182290 

Arc3-36 

5097 

3112 

347792 

6182526 

Arc3-37 

4986 

3363 

347690 

6182781 

Arc3-38 

4772 

3545 

347483 

6182971 

Arc3-39 

4602 

3754 

347321 

6183186 

Arc3-40 

4361 

3821 

347083 

6183262 

Arc3-41 

4135 

3950 

346862 

6183400 

Arc3-42 

4030 

4115 

346764 

6183569 

Arc3-43 

3781 

4138 

346516 

6183601 

Arc3-44 

3535 

4269 

346275 

6183741 

Sampler 

Number 

x  (m) 

y  (m) 

UTM-East 

(m) 

UTM-North 

(m) 

Arc2-1 

1548 

-2681 

344025 

6176872 

Arc2-2 

1690 

-2603 

344170 

6176945 

Arc2-3 

1841 

-2578 

344322 

6176964 

Arc2-4 

1998 

-2554 

344479 

6176982 

Arc2-5 

2155 

-2529 

344637 

6177001 

Arc2-6 

2313 

-2505 

344796 

6177019 

Arc2-7 

2469 

-2480 

344953 

6177038 

Arc2-8 

2616 

-2492 

345099 

6177020 

Arc2-9 

2691 

-2388 

345178 

6177121 

Arc2-10 

2747 

-2258 

345239 

6177249 

Arc2-ll 

2818 

-2134 

345315 

6177370 

Arc2-12 

2920 

-1987 

345422 

6177513 

Arc2-13 

3002 

-1830 

345510 

6177667 

Arc2-14 

3075 

-1704 

345588 

6177790 

Arc2-15 

3204 

-1629 

345720 

6177860 

Arc2-16 

3299 

-1498 

345820 

6177988 

Arc2-17 

3380 

-1367 

345906 

6178115 

Arc2-18 

3448 

-1231 

345979 

6178249 

Arc2-19 

3518 

-1093 

346054 

6178384 

Arc2-20 

3558 

-919 

346100 

6178556 

Arc2-21 

3729 

-787 

346276 

6178682 

Arc2-22 

3837 

-550 

346393 

6178914 

Arc2-23 

3998 

-418 

346559 

6179040 

Arc2-24 

4012 

-196 

346582 

6179262 

Arc2-25 

4026 

9 

346603 

6179466 

Arc2-26 

4038 

208 

346623 

6179664 

Arc2-27 

4053 

431 

346646 

6179887 

Arc2-28 

4066 

638 

346667 

6180093 

Arc2-29 

4083 

865 

346693 

6180319 

Arc2-30 

4094 

1083 

346712 

6180536 

Arc2-31 

3999 

1255 

346624 

6180712 

Arc2-32 

3891 

1459 

346524 

6180920 

Arc2-33 

3773 

1615 

346412 

6181080 

Arc2-34 

3635 

1772 

346280 

6181242 

Arc2-35 

3486 

1922 

346137 

6181398 

Arc2-36 

3304 

2079 

345961 

6181562 

Arc2-37 

3152 

2230 

345815 

6181718 

Arc2-38 

2987 

2353 

345654 

6181848 

Arc2-39 

2805 

2462 

345477 

6181964 

Arc2-40 

2628 

2543 

345303 

6182051 

Arc2-41 

2432 

2671 

345112 

6182187 

Arc2-42 

2275 

2811 

344960 

6182332 

Arc2-43 

2153 

2963 

344844 

6182489 

Arc2-20A 

3960 

-1009 

346499 

6178451 

Arc2-21A 

3972 

-818 

346518 

6178642 

Arc2-22A 

3987 

-586 

346542 

6178873 
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ADAPT  Atmospheric  Data  and  Parameterization  Tool 

AGL  above  ground  level 

ARAC  Atmospheric  Release  Advisory  Center 

BNMSE  Bounded  Normalized  Mean  Square  Error 

CBRN  Chemical,  Biological,  Radiological,  and  Nuclear 

DHS  Department  of  Homeland  Security 

DOE  Department  of  Energy 

DTRA  Defense  Threat  Reduction  Agency 

FB  Fractional  Bias 

HP  AC  Hazard  Prediction  and  Assessment  Capability 

IDA  Institute  for  Defense  Analyses 

IMAAC  Interagency  Modeling  and  Atmospheric  Assessment  Center 
LLNL  Lawrence  Livermore  National  Laboratory 


A-l 


LODI 

MANOVA 

MLOD 

MOE 

NAD 

NARAC 

NGA 

NMSE 

OLAD 

SCIPUFF 

SF6 

SWIFT 

SWIM 

T&D 


Lagrangian  Operational  Dispersion  Integrator 

multivariate  analysis  of  variance 
method  level  of  detection 
Measure  of  Effectiveness 

Normalized  Absolute  Difference 
National  Atmospheric  Release  Advisory  Center 
National  Geospatial  Intelligence  Agency 
Normalized  Mean  Square  Error 

Over-Land  Wind  Dispersion 

Second-Order  Closure  Integrated  Puff  Model 
Sulfur  Hexafluoride 
Stationary  Wind  Fit  and  Turbulence 
SCIPUFF  Weather  Input  Model 

transport  and  dispersion 
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